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Army  Project  Number 
2Q263743A794 


Education  and  Training 


FOREWORD 


The  Amy  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  perfoms  research  in  technical  training  perfom- 
ance  measurements  and  assessment  (PMA)  throughout  the  Amy,  both 
in  its  training  institutions  and  its  units.  Of  special  interest 
is  how  the  data  that  are  generated  as  a  result  of  training  PMA 
are  used  to  improve  the  training  within  the  Amy. 

This  report  provides  the  results  of  a  survey  of  a  sample  of 
six  Military  Occupational  Specialties  (MOS) .  The  survey  was 
conducted  to  detemine  how  the  training  perfomance  of  soldiers 
with  those  technical  MOS  is  measured  and  assessed  in  both  the 
institution  and  the  unit.  The  report  also  provides  insight  into 
how  the  resulting  training  PMA  data  are  used  by  the  Amy's  key 
decision  makers  to  improve  training. 

The  findings  of  this  report  confim  those  of  the  1985  Amy 
Science  Board's  Summer  Study:  The  Amy's  system  of  training  PMA 
is  often  not  providing  adequate  feedback  to  the  developers  of  the 
training  systems.  Research  to  examine  the  problems  identified  in 
this  report  could  improve  both  the  perfomance  measurement  and 
training  systems.  The  Amy  needs  a  quantifiable  method  for  tech¬ 
nical  training  perfomance  measurement  and  assessment. 

This  project  was  perfomed  under  the  research  task  called 
"Methods  for  Evaluating  Training  Systems  Effectiveness."  The 
project  supports  the  Orlando  Field  Unit's  mission  to  develop 
methods  to  optimize  simulation-based  training  systems.  It  sup¬ 
ports  the  Training  Research  Laboratory's  research  program  by 
examining  the  current  conduct  of  training  perfomance  measurement 
and  assessment  activities  in  Amy  schools  and  operational  units. 
This  research  had  two  sponsors.  One  was  the  Amy's  Project  Man¬ 
ager  for  Training  Devices  (PM  TRADE)  under  a  Memorandum  of  Under¬ 
standing  dated  18  May  1983  entitled,  "Establishment  of  Technical 
Coordination  between  ARI  and  PM  TRADE."  The  other  sponsor  was 
the  Department  of  Defense's  Training  and  Perfomance  Data  Center 
(TPDC)  under  a  Memorandum  of  Agreement  entitled,  "Amy  Research 
Institute  Coordination  with  TPDC"  dated  24  April  1985.  The  Com¬ 
manding  Officer  of  PM  TRADE  and  the  Director  of  TPDC  were  briefed 
on  the  results  of  the  project  in  September  1987.  The  proponents 
will  use  the  project's  findings  to  design  training  systems  (PM 
TRADE)  and  to  collect  training  and  perfomance  data. 
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MEASURING  AND  ASSESSING  TECHNICAL  TRAINING  PERFORMANCE  IN  THE 
ARMY'S  SCHOOLS  AND  UNITS:  A  SURVEY  OF  CURRENT  METHODS 


EXECUTIVE  SUMMARY 


Requirement: 

To  survey  some  methods  used  to  measure  individual  and  col¬ 
lective  technical  training  performance  in  the  Army's  institutions 
and  units  and  determine  how  the  resulting  data  are  used. 


Procedure : 

Six  Military  Occupational  Specialties  (MOS)  were  selected  to 
provide  a  sample  for  the  survey.  Onsite  surveys  were  conducted 
at  the  two  Service  schools  involved  in  training  soldiers  with 
those  six  MOS  and  at  units  of  III  Corps  to  which  those  soldiers 
were  assigned.  In  addition  to  the  structured  interviews,  ques¬ 
tionnaires  were  provided  to  key  decision  makers  (KDM)  in  the 
schools  to  determine  their  data  needs  and  the  extent  to  which 
those  needs  are  being  met.  No  attempt  was  made  to  survey  the  KDM 
in  units  since  the  KDMs  at  the  schools  are  responsible  for  devel¬ 
oping  the  training  and  training  support  material  used  throughout 
the  Army,  based  in  part  on  the  feedback  they  receive  from  field 
units. 

In  addition,  a  literature  review  of  recent  and  current 
research  efforts  that  address  training  performance  measurements 
and  assessment  was  done.  Current  Army  documents  describing  tne 
training  performance  measurement  and  assessment  (PMA)  policies 
and  procedures  were  also  reviewed.  The  results  of  the  review  are 
published  in  a  separate  report  entitled  "A  Review  and  Annotated 
Bibliography  of  Training  Performance  Measurement  and  Assessment 
Literature. ” 


Findings: 

The  Army  generally  develops  its  institutional  training  to  be 
responsive  to  the  needs  of  the  units  in  the  field.  While  there 
are  anomalies  in  the  system,  training  for  the  most  part,  directly 
corresponds  to  what  the  soldiers  must  do  on  the  job.  The  per¬ 
formance  of  the  individual  soldiers  and  units  is  measured  rou¬ 
tinely,  and  the  results  of  that  measurement  are  used  to  identify 
further  training  requirements.  However,  performance  is  generally 
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measured  on  a  qualitative  basis  and  is  processed  as  opposed  to 
product  oriented.  For  the  most  part,  the  resulting  data  are  not 
used  to  the  fullest.  Moreover,  there  appears  to  be  a  lack  of 
commonality  of  terms,  due  in  part  to  the  many  documents  confront¬ 
ing  the  members  of  the  training  community  and  the  imprecise  defi¬ 
nitions  of  various  training-related  terms  in  those  documents. 


Utilization  of  Findings: 

This  paper  presents  the  results  of  a  study  based  on  a  rela¬ 
tively  small  sample  of  MOS,  schools,  and  units.  The  report  pro¬ 
vides  a  basis  for  developing  more  quantitative  methods  for  PMA. 
The  findings  can  serve  as  a  springboard  for  further  study  such  as 
how  to  improve  the  flow  of  feedback  information  from  the  Units  to 
the  schools  and  how  to  improve  the  training  development  processes 
so  that  the  feedback  generated  from  the  field  is  used  in  the 
training  and  training  support  materials  developed  by  the  schools. 
At  the  very  least,  the  survey  results  and  the  literature  review 
confirm  the  previous  findings  of  the  1985  Army  Science  Board's 
Summer  Study  that  found  that  the  Army's  system  of  training  PMA 
often  does  not  provide  the  feedback  needed  by  the  developers  of 
the  training  systems.  Moreover,  the  survey  calls  attention  to 
some  lingering  problems  with  Army  training  and  measures 
performance. 
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Measuring  and  Assessing  Technical  Training  Performance 
in  the  Army's  Schools  and  Units;  A  Survey  of  Current  Methods 


OVERVIEW 


Operational  Problem 

In  order  for  the  Army's  training  system  to  be  as  responsive 
to  the  needs  of  the  Army  as  possible,  it  is  im.perative  that  there 
^e  a  means  to  measure  the  performance  of  soldiers  and  units  after 
training.  Performance  measurement  has  been  defin.id  as  the 
scoring  of  trainee  proficiency  either  subjectively  (e.g., 
instructor  opinion)  or  objectively  (e.g.,  automatic  computer 
measurement).  (MIL-HDBK-22CB. )  Performance  assessment  involves 
the  synthesis  of  all  performance  measurement  information  to 
assess  trainee  performance.  (MiL-HDBK-220B. )  This  training 
performance  measurement  and  assessment  has  a  very  important 
goal — it  provides  the  Army's  chain  of  command  with  information 
as  to  the  state  of  readiness  of  the  Army,  as  well  as  indications 
of  possible  causes  of  performance  that  are  less  than  acceptable. 
Obviously,  not  all  problems  with  substandard  performance  can  be 
traced  to  inadequate  training.  But,  certainly,  training  is  one 
of  the  factors  that  contributes  to  readiness  in  the  Army.  The 
training  performance  measurement  and  assessment  (PMA)  system, 
then,  is  a  primary  barometer  of  how  well  the  Army  trains.  The 
question  that  must  constantly  be  addressed  is  whether  training 
PMA  is  contributing  to  the  goal  of  optimizing  the  effectiveness 
of  the  Army's  training  system.  Recent  studies  such  as  the  1982 
Defense  Science  Board  and  the  Summer  Study  of  the  1985  Army 
Science  Board  suggest  that  training  PMA  is,  in  fact,  not  entirely 
effective  in  accomplishing  the  intended  goals. 

Research  Objectives 

The  objective  of  the  research  reported  herein  is  to  determine 
some  of  the  training  PMA  methodologies  in  use  in  the  Army's 
institutions  and  units  and  to  provide  an  assessment  of  how  well 
those  methodologies  contribute  to  increased  effectiveness  of  the 
Army's  training  systems.  An  additional  area  investigated 
includes  how  effectively  the  data  and  information  generated  as  a 
result  of  training  PMA  are  used  by  the  Army's  key  decision  makers 
to  make  improvements  in  the  area  of  training.  The  term  key 
decision  makers  in  this  report  is  used  to  describe  the  officials 
at  the  TRADOC  schools  who  are  responsible  for  development  of 
training  and  training  support  material  used  throughout  the  Army. 
Specifically,  the  term  applies  to  the  schools'  Directors  of 
Training  and  Doctrine. 
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Scope 

In  order  to  accomplish  the  research  objectives,  the  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
Orlando  Field  Unit  initiated  a  study  effort  to  (1)  perform  an 
extensive  literature  search  and  review;  (2)  determine  how  the 
Army  measures  performance;  (3)  determine  how  the  resulting  data 
and  information  are  used  to  improve  Army  training;  (4) 
investigate  the  cost  effectiveness  of  the  methodologies  used  to 
measure  performance;  and  (5)  assess  the  contribution  that 
training  devices  and  simulators  make  to  the  effectiveness  of  the 
training  PMA  system.  The  second  and  third  tasks  are  contained  in 
this  report.  The  remaining  tasks  are  addressed  in  two  separate 
reports  entitled  "A  Review  and  Annotated  Bibliography  of  Training 
Performance  Measurement  and  Assessment  Literature”  and  "Training 
Perfoirmance  Measurement  and  Assessment:  Technigues  for  Cost 
Effectiveness  Analysis  and  Annotated  Bibliography.” 

The  effort  reported  here  involved  a  survey  of  the 
methodologies  used  across  six  selected  MOSs  in  various  training 
situations  both  in  the  school  and  in  the  unit.  It  also  entailed 
an  identification  of  the  key  decision  makers  with  regard  to 
training  in  the  schools,  a  determination  of  their  data  and 
information  needs,  and  whether  the  results  of  the  training  PMA 
methodologies  used  satisfy  those  needs.  In  addition,  recent 
research  efforts  concerning  decision  making  were  reviewed  to 
determine  whether  other  PMA  methods  might  apply  to  the  Army's 
situation. 
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SURVEY  OF  TRAINING  PERFORMANCE  MEASUREMENT  METHODS 


The  Defense  Science  Board's  (DSB)  Suiiuner  Study  of  1982,  as 
well  as  the  Army  Science  Board's  (ASB)  1985  Summer  Study,  were 
critical  of  the  measurement  methods  used  by  the  Armed  Services 
(in  the  case  of  the  DSB)  and  the  Army  (in  the  case  of  the  ASB) . 
Specific  criticisms  pointed  to  the  lack  of  objective  standards  to 
measure  human  performance  and  the  lack  of  quantifiable  measures 
of  performance  as  two  of  the  most  serious  of  the  shortcomings. 
Other  studies  (to  be  described  later  in  this  report)  have 
investigated  the  training  PMA  in  both  the  individual  and 
collective  training  environments;  they  too  found  fault  with  some 
of  the  procedures  used  within  the  Army  to  measure  performance  in 
order  to  assess  training.  In  fact,  many  of  these  research 
efforts  produced  findings  that  echoed  the  DSB  and  ASB  Summer 
Studies.  A  recurrent  theme  in  many  of  these  studies  has  been  the 
questionable  validity  and  reliability  of  the  training  PMA  methods 
used  by  the  Army.  The  Army  Research  Institute  (ARI)  designed  the 
current  effort  to  explore  more  fully  the  problems  that  had  been 
cited,  and  to  provide  a  basis  for  suggesting  ways  by  which 
training  PMA  in  the  Army  might  be  improved.  The  first  task  of 
this  effort  was  designed  to  address  this  aspect  of  the  problem. 

This  task  involved  two  related  efforts.  The  first  was  a 
search  for  and  review  of  previous  research  efforts  that  address 
training  performance  measurement  and  assessment  (PMA) .  The 
second  was  the  survey  of  training  development  personnel  at  a 
sample  of  TRADOC  schools  and  trailers  and  training  managers  in  a 
sample  of  FORSCOM  units.  In  a<''"*icion,  survey  instruments 
designed  to  be  completed  by  dec  ion  makers  were  mailed  to  all 
the  TRADOC  schools.  The  task  was  designed  to  investigate  the 
methods  used  by  the  schools  and  units  to  measure  the  training 
performance  of  individuals  and  collectives.  The  investigation 
was  confined  to  those  training  PMA  techniques  used  to  support  the 
training  missions  in  both  the  institutional  and  unit 
environments.  Specifically  excluded  were  training  PMA  methods  to 
support  other  requirements  such  as  readiness  reporting,  although 
the  distinction  between  these  areas  is  not  always  clear. 

Methodology 

Performance  Measurement  Matrix.  The  methodology  was  designed 
to  look  at  a  small  segment  of  the  Army's  training  environment 
rather  than  at  the  entire  universe.  Accordingly,  six  MOSs  were 
selected  for  inclusion  in  the  study  effort.  The  six  MOSs  are 

19E10  -  M60  Armor  Crewman 

19K10  -  Ml  Abrams  Armor  Crewman 

63E10  -  Ml  Abrams  Tank  System  Mechanic 

63N10  -  M60  A1/A3  Tank  System  Mechanic 

67Y10  -  AHl  Attack  Helicopter  Repairer 

6BB10  -  Aircraft  Powerplant  Repairer 
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These  six  MOSs  were  chosen  as  a  result  of  discussions  among 
the  research  sponsors.  The  basis  of  the  discussions  and 
resulting  selection  was  data  availability,  mix  between  operator 
and  maintainer  MOSs,  and  cost  effectiveness  of  the  data 
collection  process.  The  six  MOSs  represent  a  mix  between 
operator  MOSs  and  maintainer  MOSs.  Four  armor-related  MOSs  were 
selected  so  that  the  data  collection  phase  of  the  effort  would  be 
more  cost  effective;  since  data  relating  to  institutional 
training  of  the  four  armor  MOSs  could  be  gathered  at  one 
installation — Fort  Knox — considerably  less  coordination  and 
travel  would  be  required  than  if  four  different  institutions  were 
involved.  The  aviation  maintenance  MOSs  were  selected  on  the 
basis  that  these  MOSs  might  be  generalizable  to  the  other 
Services.  Moreover,  these  two  MOSs  are  trained  at  one  TRADOC 
school,  the  Aviation  Logistics  School  at  Fort  Eustis,  making  data 
collection  relatively  more  cost  effective.  Finally,  it  was 
believed  that  all  six  of  these  MOSs  could  be  found  at  one 
installation — Fort  Hood,  Texas — which  would  again  make  the 
collection  of  data  concerning  training  PMA  in  units  more  cost 
effective. 

A  matrix  (Figure  1)  provided  the  framework  for  the  collection 
of  data  concerning  training  PMA.  This  data  collection  took  place 
in  the  schools  (institutions)  that  train  the  MOSs  selected  and  in 
the  units  to  which  soldiers  with  those  MOSs  are  assigned.  Along 
the  vertical  axis  of  the  matrix  are  arrayed  the  six  MOSs 
selected.  The  horizontal  axis  is  divided  into  institutional 
training  and  unit  training.  Institutional  training  is  further 
divided  into  individual  and  collective  training.  Unit  training 
is  likewise  divided  into  individual  and  collective  training;  in 
addition,  individual  training  in  units  is  further  subdivided  into 
categories  of  training  identified  by  the  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC)  as  discrete  elements,  to  wit: 
integration  training,  sustainment  training,  and  enhancement 
training. 

The  selection  of  the  categories  of  individual  training  in 
units  was  to  a  certain  extent  arbitrary  in  light  of  conflicting 
guidance  contained  in  a  variety  of  publications.  For  example,  AR 
350-1,  Army  Training,  states  in  part  that  Individual  Training  "is 
taught  in  units  to  sustain  skills  taught  in  the  training  base, 
teach  those  skills  not  taught  in  the  training  base,  and  through 
collective  training,  prepare  the  soldier  to  become  a  skilled 
member  of  his  or  her  unit."  TRADOC  Regulation  350-7,  a  Systems 
Approach  to  Training,  cites  the  following  components  of 
individual  training  in  units;  integration,  train  up,  sustainment, 
transition,  merger,  and  crosstrain.  The  TRADOC  Primer  cites  the 
following  components  of  individual  training  in  units: 
integration  training,  sustainment  training,  train-up  training, 
and  cross  training.  In  order  to  keep  the  matrix  as 
concise  and  usable  as  possible,  discussions  were  held  with  staff 
officers  within  the  Office  of  the  Deputy  Chief  of  Staff  for 
Training  (ODCST)  at  TRADOC  to  solicit  their  views  on  the  meanings 
of  the  various  categories  contained  in  the  publications  governing 
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19E10  M60  Armor  Crewman 


19K10  Ml  Abrams  Armor  Crew 
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individual  training  in  units.  Based  on  those  discussions,  it  was 
decided  to  group  train-up,  cross-training,  transition  training, 
and  merger  training  into  a  single  category  collectively  called 
enhancement  training.  Accordingly,  those  three  categories  were 
incorporated  into  the  matrix.  The  definitions  of  those 
categories  is  as  follows: 

Integration  Training;  The  training  a  newly-assigned  soldier 
needs  before  he/she  can  become  a  productive  member  of  the  unit. 
This  may  include  task-related  training  the  soldier  has  already 
received  in  the  institution  or  it  may  involve  training  not 
previously  received.  According  to  the  TRADOC  Primer,  soldiers  in 
the  Active  Component  (AC)  "should  be  fully  integrated  within 
three  months  of  arrival"  in  the  units;  those  in  the  Reserve 
Component  (RC) ,  within  one  year." 

Sustainment  Training;  Training  the  individual  soldier 
receives  to  sustain  proficiency  in  skills  that  have  previously 
been  learned,  either  in  the  institutional  or  the  unit  training 
environment.  The  TRADOC  Primer  states  that  the  "frequency  with 
which  it  is  conducted  will  vary  with  individual  and  collective 
tasks,  the  role,  location,  and  personnel  fill  of  the  units,  as 
well  as  the  desires  of  the  commander." 

Enhancement  Training:  Training  the  soldier  has  not 
previously  been  exposed  to  but  which  is  required  as  a  result  of 
the  introduction  of  new  equipment;  a  change  in  doctrine,  tactics, 
or  procedures;  to  enable  the  soldier  to  perform  the  duties  of 
other  members  of  his/her  unit;  or  to  prepare  the  soldier  for 
promotion  up  the  career  ladder. 

The  completed  matrices,  filled  with  the  data  gathered  from 
the  survey  of  six  MOSS,  are  exhibited  in  Appendix  D.  Each  cell 
of  the  matrix  shows  the  number  of  tasks  trained  and  evaluated, 
the  standards  that  are  used,  the  training  PMA  methodology 
employed,  and  the  use  of  the  resulting  data  by  trainers,  training 
managers,  and  key  decision  makers.  In  addition,  the  completed 
matrix  provides  insights  to  the  generalizability  of  training  PMA 
among  the  various  MOSs,  and  by  extension  other  similar  MOSs,  in 
the  various  training  situations  depicted.  The  cells  also  provide 
a  convenient  means  of  comparing  the  survey  data  among  the  six 
surveyed  MOSs. 

A  literature  search  was  conducted  employing  computer-assisted 
and  manual  approaches.  The  various  reports  that  were  accessed 
and  reviewed  provided  valuable  insights  into  the  nature  of  the 
problems  associated  with  training  PMA.  Although  the  relevant 
findings  and  conclusions  of  these  reports  are  addressed  in  this 
report,  the  detailed  results  of  the  literature  search  are 
included  in  a  separate  report. 

Survey .  Contact  was  made  with  the  schools  which  provide  the 
institutional  training  for  the  six  MOSs:  the  Armor  School  at 
Fort  Knox,  KY  for  MOS  19E10,  lOKlO,  63E10,  and  63N10,  and  the 
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Aviation  Logistics  School  at  Fort  Eustis,  VA,  for  MOS  67Y10  and 
68B10.  Except  for  MOS  63E10  and  63N10,  these  schools  are  also 
the  proponent  for  the  design  of  the  institutional  training.  (In 
those  instances,  although  the  Armor  School  actually  presents  the 
instruction  and  trains  the  MOS,  it  is  the  Ordnance  Center  and 
School  that  develops  the  curriculum  and  training  materials  for 
the  courses.)  The  two  schools  involved  in  the  actual  training — 
the  Armor  School  and  the  Aviation  Logistic  School — were  visited 
for  the  purpose  of  explaining  the  scope  of  the  study  effort  and 
soliciting  their  support.  Personnel  contacted  during  the 
original  visits  were  Mr.  Gary  Priest  of  the  Directorate  of 
Training  and  Doctrine  of  the  Armor  School,  and  Mr.  Andy  Davis  of 
the  Directorate  of  Training  and  Doctrine  of  the  Aviation 
Logistics  School. 

Following  the  initial  visits  to  the  two  schools,  arrangements 
were  made  to  conduct  structured  interviews  with  personnel 
selected  by  the  schools.  The  schools  were  requested  to  select 
only  those  personnel  who  would  be  likely  to  have  the  kinds  of 
information  that  would  be  meaningful  to  the  study  effort.  To 
this  end,  a  draft  survey  instrument  was  provided  the  schools 
during  the  initial  visit  in  order  to  outline  for  them  the  kinds 
of  information  being  sought.  Each  school  then  drew  up  a  list  of 
personnel  they  believed  would  be  most  responsive  and  made 
arrangements  for  those  personnel  to  meet  with  the  survey  team 
during  a  subsequent  visit. 

During  the  subsequent  visit  to  each  school,  members  of  the 
survey  team  conducted  in-depth  structured  interviews  with  the 
respondents  identified  by  the  school.  The  survey  instrument  used 
during  the  interviews  is  included  in  Appendix  A. 

Once  the  data  were  collected  on  institutional  training  and 
training  PMA,  efforts  were  undertaken  to  capture  similar  data  at 
the  units  to  which  soldiers  with  the  MOSs  under  study  were 
subsequently  assigned.  Because  Fort  Hood,  TX  has  such  a  large 
armor  and  aviation  community,  it  was  selected  as  the  most  likely 
installation  for  meeting  the  data  needs  of  the  study  effort; 
moreover,  if  all  the  MOSs  were  located  at  a  single  installation, 
it  would  result  in  a  conservation  of  travel  funds.  Accordingly, 
the  Chief  of  Staff's  Office  at  III  Corps  and  Fort  Hood  was 
contacted  to  determine  the  feasibility  of  conducting  structured 
interviews  with  personnel  who  would  be  in  the  best  position  to 
provide  the  kind  of  information/data  needed.  As  with  the 
schools,  in  order  to  assist  the  units  in  making  that 
determination,  a  draft  survey  instrument  was  provided. 

Ill  Corps  indicated  its  willingness  to  participate  in  the 
survey,  and  the  Study  Group  in  the  Office  of  the  III  Corps  Chief 
of  Staff  arranged  for  members  of  the  survey  team  to  meet  with 
selected  platoon  sergeants,  platoon  leaders  and  company 
commanders  in  units  to  which  the  soldiers  with  the  appropriate 
MOSs  were  assigned.  At  the  time  the  MOSs  were  selected.  III 
Corps  had  units  associated  with  each  of  those  MOSs;  thus  the 
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rationale  for  selecting  these  MOSs  appeared  sound  and  data  were 
collected  during  the  institutional  surveys.  However,  two  weeks 
prior  to  the  arrival  of  the  survey  team  at  Fort  Hood,  the  last  of 
the  M60  tank  battalions  were  reorganized  as  Ml  Abrams  battaJions, 
and  it  was  no  longer  possible  to  interview  the  supervisors  of  MOS 
19E10  and  63N10  soldiers.  This  development  notwithstanding, 
discussions  between  the  research  staff  and  the  contract  officer's 
technical  representative  (COTR)  resulted  in  the  decision  to  keep 
the  two  M60-rel«ted  MOSs  in  the  data  collection  matrix  even 
though  data  concerning  the  training  performance  measurement  of 
those  two  MOSs  in  the  unit  environment  would  not  be  available. 

The  rationale  for  this  decision  was  that  valuable  information 
concerning  performance  measurement  in  the  institution  had  been 
gathered  and  should  not  be  discarded.  There  was  also  some 
discussion  as  to  the  viability  of  expanding  the  survey  to  include 
other  units  that  still  had  M-60  tanks  in  their  inventory. 

However,  given  the  extensive  lead  time  required  to  arrange 
surveys  in  the  Army's  field  units,  this  idea  was  rejected. 

Arrangements  for  the  data  collection  trip  to  Fort  Hood  were 
made  and  structured  interviews  were  conducted.  The  survey 
instrument  used  during  these  interviews  is  provided  at 
Appendix  B.  Following  preliminary  analysis  of  the  type  and 
amount  of  data  generated  as  a  result  of  the  initial  survey,  the 
COTR  decided  to  expand  the  sample  thereby  increasing  the  amount 
of  data  on  which  the  findings  and  conclusions  of  Lae  report  would 
be  based.  This  preliminary  analysis  also  provided  greater 
insight  as  to  the  type  of  knowledge  the  respondents  could 
reasonably  be  expected  to  have.  In  light  of  this,  the 
questionnaire  was  changed  slightly,  eliminating  some  of  the 
questions  in  the  first  and  modifying  some  others.  (This  second 
questionnaire  is  provided  at  Appendix  C.)  In  order  not  to 
invalidate  the  results  from  the  first  questionnaire,  the  data 
from  both  were  pooled  to  the  extent  possible. 

Additional  information  relative  to  the  perceived 
effectiveness  of  the  Army's  PMA  system  in  influencing  the 
development  of  training  programs  was  solicited  from  a  variety  of 
other  agencies  with  responsibilities  in  training  and  performance 
measurement  during  the  study.  These  agencies  generally  have  a 
broader  perspective  of  the  PMA  methods  used  across  units, 
schools,  and  MOSs  although  the  level  of  detail  is  not  as  great. 
The  impressions  resulting  from  discussions  with  officials  at 
these  agencies  are  generally  in  accord  with  the  results  of  the 
surveys  conducted  at  the  two  schools  and  the  units  of  III  Corps. 
The  agencies  contacted  included  the  Army  Training  Board  (ATB)  and 
the  Office  of  the  Deputy  Chief  of  Staff  Training  (ODCST)  of  the 
Army  Training  and  Doctrine  Command  (TRADOC)  at  Fort  Monroe, 
Virginia;  the  TRADOC  Systems  Analysis  Agency  (TRASANA)  at  White 
Sar^ds  Missile  Range,  New  Mexico;  the  Combined  Arms  Training 
Activity  (CATA)  at  Fort  Leavenworth,  Kansas;  the  Florida  National 
Guard  at  St.  Augustine,  Florida;  and  Readiness  Group  Redstone,  at 
Redstone  Arsenal,  Alabama. 


Following  the  collection  of  data  generated  as  a  result  of  the 
literature  review,  the  interviews  conducted  in  the  schools  and 
units,  and  the  discussions  held  with  officials  at  the  agencies 
mentioned  in  the  preceding  paragraph,  the  data  were  reduced  and 
analyzed.  These  data  have  been  used  in  formulating  the  results 
and  findings  of  this  section  of  the  report. 

Results/Findinas 

The  following  is  a  discussion  of  the  results  of  the  survey  of 
training  PMA  as  it  is  practiced  in  the  Army,  it  was  developed 
from  the  interviews  with  respondents  at  the  Armor  School, 

Aviation  Logistics  School,  and  units  of  III  Corps.  The 
respondents  at  the  two  schools  were  military  and  civilian 
officials  involved  in  the  analysis  and  design  of  the  training 
programs  used  for  training  individual  soldiers  in  the 
institutional  environment  and  preparing  the  material  used  to 
support  individual  and  collective  training  in  the  units.  The 
respondents  in  the  IIJ  Corps  units  were  officers  and  non¬ 
commissioned  officers  a.nvolved  in  individual  and  collective 
training  in  the  unit  environ — ^nt;  without  exception  they  held 
leadership  positions  at  the  platoon,  company  and  battalion  level. 
The  survey  sample  does  not  permit  ready  generalizations  across 
all  MOSs,  schools,  and  units.  The  validation  of  these  findings, 
to  an  acceptable  degree  of  confidence,  would  therefore  require 
further  research.  However,  it  would  appear  from  discussions  with 
survey  respondents  and  officials  at  the  various  source  agencies, 
including  those  discussed  in  the  preceding  section,  that  a 
reasonably  strong  inference  can  be  made  that  similar  findings 
would  be  likely  in  most,  if  not  all,  of  the  MOSs  involved  in  this 
study  effort.  The  survey  validated  the  findings  of  the 
literature  review. 

In  the  interests  of  brevity,  this  section  presents  only  an 
overview  of  the  pertinent  findings  and  conclusions  of  the 
school/unit  survey.  The  reader  wishing  to  read  the  details  of 
the  survey  and  the  rationale  for  arriving  at  the  various 
findings/conclusions  is  referred  to  Appendix  D  of  this  report. 

The  literature  that  was  accessed  and  reviewed  is  addressed  in  a 
separate  report. 


Quantification  of  Performance  Measurement  Results.  Although 
the  previously-referenced  Army  Science  Board's  Summer  Study  of 
1985  cited  a  need  to  increase  quantification  of  training 
objectives,  tasks,  and  measurement  programs,  to  include  data  on 
skill  retention  and  learning  rates,  there  appears  to  have  been 
little  progress  made  to  accomplish  this.  For  the  most  part,  the 
Army  uses  qualitative  criteria  to  assess  the  effectiveness  of  its 
individual  and  collective  training  programs.  As  a  result,  the 
reliability  of  the  data  resulting  from  training  performance 
measurement  in  the  Army  is  in  some  cases  questionable,  and  the 
most  frequently  cited  weaknesses  of  the  measurement  system  appear 
to  remain. 


Relationship  of  Tasks  Trained  to  Job/Mission  Requirements. 
Although  pertinent  regulations  such  as  TRADOC  Regulation  350-7,  A 
Systems  Ap.jroach  to  Training,  require  that  tasks  trained  in  the 
institution  be  developed  based  on  an  analysis  of  the  individual 
and  collective  missions  and  tasks  required  in  the  job 
environment,  there  are  indications  that  this  is  not  being 
accomplished  in  accordance  with  established  policy.  The 
collective  front-end  analysis  (CFEA)  is  designed  to  analyze  a 
particular  unit's  mission  and  from  that  develop  the  collective 
and  individual  tasks,  skills  and  knowledges  that  will  allow  for 
mission  accomplishment.  Discrepancies  between  institutional 
Programs  of  Instruction  and  the  pertinent  soldier's  manual 
suggest  that  there  is  a  disconnect  between  what  is  trained  in  the 
school  and  what  is  required  of  the  soldier  on  the  job.  This 
apparent  disconnect  is  supported  by  an  article  by  senior 
officials  at  TRADOC  (Duncan  &  Hartjen,  1985)  in  which  the  authors 
indicate  that  front-end  analyses  are  not  being  conducted  in 
accordance  with  policy  guidelines  in  TRADOC  Reg  350-7. 

Process  vs.  Product  Orientation.  The  Army,  in  its  approach 
to  measuring  individual  training  performance,  orients  almost 
exclusively  on  tne  'process'  involved  in  task  performance  as 
opposed  to  the  'product.'  While  there  is  evidence  to  support  the 
use  of  such  'process'  orientation  in  an  institutional  training 
environment,  especially  where  entry  level  training  is  being 
conducted,  the  research  suggests  that  its  extensive  use  in  a  unit 
training  environment  may  be  a  questionable  practice.  This 
notwithstanding,  the  survey  respondents  indicated  that,  as  a 
general  rule,  individual  performance  measurement  in  the  unit 
training  environment  is  process  rather  than  product  oriented.  In 
both  institutional  and  unit  training,  individual  task  training 
PMA  is  generally  based  on  either  the  soldier's  manual  or  a 
technical  manual.  These  documents  stress  the  procedures  the 
soldier  must  follow  in  order  to  get  a  GO  on  the  overall  task. 

None  of  the  survey  respondents  indicate  that  product-oriented 
training  PMA  is  extensively  used. 

With  respect  to  collective  training  in  units,  there  is  a 
greater  tendency  to  focus  on  the  product  rather  than  the  process. 
Survey  respondents,  especially  those  involved  with  armor  crew 
training,  note  that  in  many  cases  process  is  less  important  than 
product.  The  most  often  cited  example  was  in  gunnery  training, 
in  which  the  respondents  indicate  that  the  important  performance 
criterion  is  putting  steel  on  the  target  rather  than  the  gunnery 
procedures  per  se. 

Criterion-  vs.  Norm-Referenced  PMA.  Criterion-referenced  PMA 
compares  the  trainee's  performance  to  a  pre-established  standard. 
The  performance  of  a  trainee's  fellow  trainees  does  not  affect 
the  rating  of  his  performance  under  a  criterion-referenced 
approach.  Conversely,  under  a  norm-referenced  PMA  strategy  the 
trainee's  performance  rating  is  based  upon  the  performance  of  his 
peers.  The  standard  for  an  acceptable  PMA  rating  for  the  trainee 
is  based  upon  how  well  the  top  performers  in  his  training  class 
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do.  There  is  virtually  no  norm-referenced  training  PMA  being 
used  in  the  various  training  programs  for  skill  level  1  soldiers 
of  the  six  surveyed  MOSs.  What  little  norm-referenced  training 
PMA  there  is,  is  confined  to  the  schools,  and  even  in  that 
environment  it  amounts  to  only  a  small  percentage.  In  all 
instances  of  norm-referenced  measurement,  written  tests  are 
involved;  70%  is  generally  the  passing  grade.  The  use  of  written 
tests,  and  norm-referenced  training  PMA,  appears  to  be  limited  to 
subjects  dealing  with  filling  out  forms  and  records. 

Use  of  Feedback  Data.  Although  the  survey  respondents 
without  exception  acknowledged  the  importance  of  the  feedback 
loop  between  the  units  in  the  field  and  the  TRADOC  schools 
responsible  for  providing  trained  soldiers  and  training  support 
material  to  the  field  units,  there  appears  to  be  a  breakdown  in 
the  Army's  ability  to  make  that  loop  effective.  While  it  is  true 
that  TRADOC  mandates  that  schools  be  proactive  in  accessing 
performance  measurement  data  generated  in  the  unit  training 
environment,  none  of  the  survey  respondents  in  the  units  have  had 
any  direct  contact  with  representatives  of  the  school  having 
proponency  for  the  type  unit  to  which  they  are  assigned.  It  is 
recognized  that,  due  to  the  limited  number  of  respondents 
surveyed  during  this  effort,  the  conclusions  drawn  may  be  the 
result  of  sampling  error.  On  the  other  hand,  if  the  survey 
results  are  in  fact  valid  and  generalizable  to  other  MOSs,  this 
constitutes  a  potentially  serious  problem  for  the  Army,  since  the 
effectiveness  of  the  Systems  Approach  to  Training  (SAT)  depends 
on  the  maintenance  of  an  effective  channel  of  communication 
between  the  units  for  which  training  and  training  support 
material  is  developed  and  the  TRADOC  school  doing  the 
development. 

Lack  of  Standardized  Terminology.  While  less  serious  perhaps 
than  the  problems  discussed  above,  the  authors  noted  that 
communication  regarding  training  and  training  performance 
measurement  is  hampered  by  a  lack  of  clear  definitions  of  the 
temms  used  by  those  involved  in  the  training  and  training 
development  communities.  Terms  used  in  the  TRADOC  schools  are 
often  misunderstood  by  trainers  in  the  units;  on  other  instances, 
trainers  in  the  units  use  terms  that  are  different  from  terms 
used  in  TRADOC 's  training  support  materials.  In  addition,  there 
are  too  many  terms  used,  resulting  in  confusion  on  the  part  of 
many  involved  in  the  training  process.  This  lack  of  a  common 
terminology  is  seen  as  contributing  to  the  lack  of  communication 
between  TRADOC  and  the  users  of  its  products. 

Discussion 


The  findings  outlined  above  lead  to  some  obvious  conclusions. 
These  are  discussed  below,  and  generally  reflect  the  opinions  of 
the  authors. 

Despite  problems  that  exist  within  the  Army  with  regard  to 
the  measurement  of  performance — problems  that  have  been  addressed 
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in  detail  in  the  Army  Science  Board's  1985  Summer  Study — the  Army 
would  appear  to  be  making  some  progress  in  coming  to  grips  with 
the  deficiencies  of  the  system.  Despite  the  gains  made,  there 
are  areas  in  which  much  more  progress  needs  to  be  made.  Some  of 
these  are  addresed  below: 

Reliability  and  Validity.  Although  very  little  hard  data 
were  available  to  the  researchers  to  make  assessments  of  the 
reliability  and  validity  of  the  training  PMA  methodologies 
currently  in  use  and  the  resulting  data,  the  survey  respondents 
were  asked  to  provide  their  own  assessments  with  regard  to  those 
areas.  There  were  few  differences  between  the  respondents  in  the 
school  and  those  in  the  units.  Generally,  these  respondents 
believed  that  the  methodologies  used  to  evaluate  individual 
performance  were  valid  and  evaluated  what  they  were  intended  to. 
Moreover,  they  believed  that  the  results  obtained  from  different 
iterations  of  an  evaluation  were  generally  consistent  over  time. 
In  the  absence  of  statistical  data,  however,  these  assessments 
must  be  viewed  with  a  certain  amount  of  caution,  since  in  many 
cases  the  researchers  had  to  provide  definitions  of  the  terms 
before  the  respondents  could  answer.  It  may  be  said  then  that, 
to  the  extent  the  methodologies  and  data  were  claimed  to  be 
reliable  and  valid,  the  respondents  could  only  provide 
indications  of  face  reliability  and  validity. 

The  authors  believe  that,  given  the  fact  that  many  of  the 
product-oriented  standards  used  in  the  Army's  current  training 
materials  are  qualitative  and  too  subjective,  a  situation  that 
numerous  research  efforts  (e.g.,  Havron  &  McFarling,  1979;  Hayes 
&  Wallis,  1979)  have  reported  on,  an  argument  can  be  made  that 
reliability  generally  increases  the  more  the  methodology  is 
process-oriented.  Thus,  although  a  product-oriented  approach  has 
advantages,  there  could  be  unacceptable  tradeoffs  in  terms  of 
reliability  were  the  Army  to  adopt  a  more  product-oriented 
approach. 

Process  vs .  Product .  The  Army's  concentration  on  process 
oriented  PMA  is  to  a  certain  extent  a  philosophical  issue  and 
possibly  reflects  the  personal  inclinations  of  the  TRADOC 
Commander.  When  criterion-referenced  training  was  first  being 
introduced  to  the  TRADOC  school  system  in  the  early  1970s,  the 
Commanding  General  of  TRADOC  encouraged  school  commandants  to 
provide  standards  that  focused  on  whether  a  soldier  met  the 
particular  criteria  and  not  whether  he  performed  all  of  the 
procedural  steps  along  the  way.  In  the  case  of  the  boresighting 
example  discussed  in  detail  in  Appendix  D,  a  soldier  would 
receive  a  GO  if  the  evaluator  determined  that  the  center  of  the 
barrel  and  the  center  of  the  infinity  sight  reticle  were  in  fact 
on  the  same  aiming  point.  The  soldier's  ability  to  perform  the 
intervening  steps  was  assumed  if  he  met  the  standard.  Obviously, 
if  the  soldier  failed  to  align  the  barrel  and  the  sight  reticle 
as  established  in  the  standard,  there  had  to  be  a  basis  for 
critiquing  the  failure  and  explaining  to  the  soldier  why  he 
failed.  Thus,  even  while  TRADOC  was  emphasizing  product,  a 


certain  degree  of  process-orientation  was  evident  in  the  training 
PMA  system  being  developed  by  TRADOC  schools. 

In  the  authors'  opinion,  the  concentration  on  "process,” 
especially  in  the  unit  training  environment,  is  a  questionable 
practice.  Given  the  fact  that  it  is  more  costly  (in  terms  of 
time  required  and  number  of  evaluators  required)  to  evaluate 
process  as  opposed  to  product,  it  probably  would  make  better 
sense  to  measure  training  performance  in  teirms  of  the  ability  of 
the  soldier  or  unit  to  meet  the  performance  standard  rather  than 
evaluate  how  the  standard  was  met. 

Criterion-Referenced  PMA.  There  is  no  margin  for  error  in 
the  institutional  training  environment  for  those  subject  areas 
that  are  measured  using  the  criterion-referenced  approach.  Not 
only  must  the  students  pass  each  procedure  in  a  task  to  get  a  GO 
on  the  entire  task,  they  must  get  a  GO  on  every  task  in  order  to 
complete  the  course.  The  two  schools  involved  in  the  survey  have 
policies  which  require  that  if  a  soldier  gets  a  NO  GO  on  a  task 
the  first  time  his  performance  is  measured,  he  receives 
additional  training,  often  one-on-one  with  the  instructor,  until 
he  can  perform  to  standard.  According  to  most  of  the  survey 
respondents,  if  this  intensive  remedial  training  fails  to  produce 
a  soldier  who  can  perform  the  task,  the  soldier  is  reclassified 
to  another  MOS  in  which  the  he/she  can  perform  or  is  boarded  out 
of  the  Army.  These  actions  however  are  taken  only  as  a  last 
resort . 

Uses  of  Performance  Measurement  Data.  Data  generated  as  a 
result  of  training  PMA  are  used  in  a  variety  of  ways.  In  the 
institutional  setting,  they  are  used  to  determine  whether  the 
student  is  able  to  perform  a  particular  task  to  the  prescribed 
standard  and  whether  remedial  training  is  called  for.  They  are 
also  used  to  provide  indications  of  the  effectiveness  of  the 
instruction.  For  example,  at  the  Armor  school  if  there  is  a  20% 
or  higher  failure  rate  on  a  particular  task,  the  Department  of 
Evaluation  and  Standardization  is  required  to  investigate  the 
reasons  for  that  high  rate.  While  some  of  the  survey  respondents 
indicate  that  data  concerning  first  attempts  at  task 
accomplishment  are  captured  and  used  as  a  quality  control  check, 
other  respondents  state  that  those  data  are  generally  not 
available. 

In  the  unit  training  environment,  individual  task  training 
PMA  data  are  also  used  to  determine  if  the  soldier  meets  the 
prescribed  standard  of  performance.  The  determination  is  used  by 
the  unit's  trainers  and  training  managers  to  make  decisions  as  to 
future  training  requirements,  i.e.,  whether  to  conduct  remedial 
training  or  to  move  on  to  training  on  other  tasks.  The  data  are 
also  used  by  the  command  structure  to  infer  the  effectiveness  of 
the  unit's  individual  training  program.  Finally,  the  training 
PMA  data  generated  from  the  administration  of  the  skill 
qualification  tests  are  used  for  various  personnel  actions,  such 
as  promotion,  reclassification,  reassignment,  and  board  actions. 
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with  regard  to  collective  task  training  PMA,  data  are 
collected  and  used  to  evaluate  the  unit's  ability  to  accomplish 
its  wartime  and  contingency  missions  and  to  infer  the 
effectiveness  of  the  unit's  collective  training  program.  In 
addition,  the  data  are  used  to  feed  the  readiness  reporting 
system,  a  system  that  is  used  by  commanders  and  staffs  at  higher 
levels  of  command. 

Recommendations 

While  performance  measurement  data  is  almost  universally 
recognized  as  being  extremely  important  to  improving  the  training 
conducted  within  the  Army's  schools  and  units,  there  does  not 
appear  to  be  any  comprehensive  system  for  ensuring  that  those 
data  are  used  to  accomplish  that  goal.  Specifically,  research  is 
needed — and  the  results  of  the  research  analyzed  and  acted  on — in 
the  following  areas; 

Methods  of  enhancing  the  objectivity  of  performance 
measurement,  either  through  increased  quantification  of 
the  measurement  system,  improved  standards  or  both. 

Methods  for  improving  the  linkage  between  what  the 
soldiers  must  do  on  the  job  (individually  and  as  part  of 
a  unit)  and  what  they  are  trained  to  do  in  both  the 
institutional  and  unit  training  environments. 

Identification/selection  of  areas  or  domains  where 
process-oriented  vs.  product  oriented  PMA  should  be 
employed. 

Identification  of  the  most  cost-effective  means  for 
ensuring  that  data  resulting  from  PMA,  whether  in  the 
institution  or  unit,  are  used  to  influence  changes  to 
the  training  received  by  soldiers  and  units. 

Analysis  of  the  extent  to  which  all  of  the  procedural 
steps  included  in  training  support  material — and  which 
are  subject  to  performance  measurement — are  in  fact 
necessary  for  job/mission  accomplishment. 

Analysis  of  the  extent  of  confusion  resulting  from  the 
large  number  of  often  contradictory  regulations 
governing  training  and  PMA. 

Methods  for  improving  the  reliability  and  validity  of 
the  PMA  system  throughout  the  Army. 
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KEY  DECISION  MAKER  SURVEY 


The  preceding  sections  of  this  report  (and  a  separate  report 
devoted  to  the  literature  review)  have  described  the  general 
state  of  the  art  of  performance  measurement  and  assessment  (PMA) . 
Methods  by  which  it  is  currently  addressed  in  today's  Army  were 
alsc  explored. 

The  state  of  the  art  survey  did  not  reveal  any  literature 
which  addressed  Army  Key  Decision  Maker  (KDM)  performance 
measurement  and  assessment  concerns.  This  void  makes  it 
difficult  to  draw  conclusions  about  the  effectiveness  of  the 
Army's  current  system.  How  can  we  know  if  the  PMA  systems  meet 
Army  requirements  if  we  don't  know  what  those  requirements  are? 

The  1985  Army  Science  Board's  findings  addressed  this  probxem 
of  KDMs  and  training  PMA  in  training.  The  Board  stated  that, 
"measurement  of  training  is  necessary  to  provide:  feedback  to 
trainers  and  training  designers,  and  Return  on  Investment 
information  to  senior  managers  to  guide  expenditure  of  Army 
training  resources."  (p.  29).  These  'trainers'  and  'senior 
managers'  are  those  identified  as  KDMs  in  this  study.  In  order 
to  keep  the  study  focused  as  much  as  possible  on  training,  the 
KDMs  surveyed  were  those  who  manage  the  Directorates  of  Training 
and  Doctrine  at  the  major  Army  schools. 

To  fill  the  information  void  about  KDMs  described  above,  a 
methodology  was  devised  for  determining  their  PMA  requirements. 
Key  questions  addressed  were: 

•  Who  are  the  KDMs  in  the  Army's  schools,  with  regard  to 
training  PMA? 

•  What  are  their  key  information  requirements? 

•  Are  they  currently  receiving  the  information  necessary 
to  satisfy  their  requirements? 

•  Is  the  information  in  qualitative  or  quantitative  form? 
(A  major  concern  of  the  1985  Army  Science  Board.) 

•  Which  information  is  most  and  least  useful  for  the 
decisions  the  KDMs  have  to  make? 

Methodology 

The  methodology  adapted  for  the  survey  combined  elements  of 
Policy  Capturing  Analysis  (Madden,  1963)  with  elements  of  Policy 
Implications  Analysis  (Madey  and  Stenner,  1981)  and  the  Delphi 
Technique  (Dalkey,  1969) .  These  techniques  are  used  by 
management  analysts,  industrial-organizational  psychologists,  and 
evaluation  specialists  to  determine  critical  aspects  of  policy 
development  and  information  use  by  KDMs. 
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I 

An  abbreviated  form  of  the  methodologies  described  above  was 
adapted  for  the  KDM  survey  for  two  reasons.  First,  those 
methods  described  above  are  primarily  intended  to  aid  analysts  in 
predicting  future  actions  of  managers  and  experts  in  policy 
making  questions.  This  study  was  done  for  descriptive,  rather 
than  predictive  purposes.  The  intent  was  to  capture  current 
concerns  and  interests  of  KDMs,  while  identifying  training  PMA 
issues  for  further  study.  The  methodologies  were  also  modified 
to  reduce  the  time  demands  on  respondents.  Initial  contacts  with 
some  of  the  respondents  indicated  that  their  schedules  were 
extremely  tight.  The  researchers  determined  that  administration 
of  the  more  formal  methodologies  may  not  have  resulted  in  an 
adequate  response  rate  had  the  respondents  felt  too  much  time  was 
required. 

Two  questionnaires  were  developed.  The  first  (Appendix  E) 
was  intended  to  query  KDMs  about  their  training  PMA  information 
needs.  The  questionnaire  was  sent  out  to  twenty-three  schools 
within  the  Army,  of  which  nine  responded.  In  the  second 
questionnaire  (Appendix  F)  consensus  responses  were  developed  for 
the  key  PMA  items  explored  in  the  first  questionnaire.  Again,  to 
obtain  the  highest  response  rate  possible,  and  in  the  interest  of 
the  respondents'  time,  the  instrument  was  kept  brief.  The 
instructions  asked  that  the  Director  of  Training  and  Doctrine 
(DOTD)  complete  the  questionnaire.  The  goal  of  the  consensus 
instrument  was  to  have  individuals  respond  who  are  truly  KDMs. 

The  results  of  the  first  inquiry  had  shown  that  the  DOTD  were 
considered,  by  all  schools  responding,  to  be  the  KDMs.  A  copy  of 
the  first  questionnaire  was  included  in  the  mailing  of  the  second 
so  that  any  new  respondents  could  understand  the  context  of  the 
study.  Eleven  second  round  responses  were  received  from  the 
schools  which  participated.  Included  in  this  response  were  three 
major  integrating  centers;  Combined  Arms  Center,  Logistics 
Center,  and  Soldier  Support  Center. 

Results/Findinas 

This  section  is  organized  around  the  two  questionnaires  which 
were  used  to  collect  data  from  the  KDMs.  Tables  are  provided  for 
the  results  of  each  question.  The  first  question  from  the  first 
questionnaire  asked  the  respondents,  "Who,  at  the  school,  makes 
decisions  on  training  performance  measurement  and  assessment?" 

Every  response  indicated  that  the  Director  of  Training  and 
Doctrine  makes  these  decisions.  There  also  were  a  few  responses 
which  mentioned  the  Director  of  Evaluation  and  Standardization  as 
an  additional  decision  maker. 

Table  1  shows  the  results  of  the  second  question  from  the 
first  questionnaire.  This  question  was  intended  to  show  how  much 
impact  the  KDMs  have  on  PMA  issues. 
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TABLE  1. 


Effect  of  KDM  decisions. 

Question  2.  How  often  do  the  KDM's  decisions  affect  the  following? 

Note:  The  responses,  in  some  cases,  exceed  the  number  of 
questionnaires  returned,  because  some  respondents  gave  more  than  one 
answer. 


Never  Seldom 


Sometimes  Generally  Always 


a.  What  is  measured  3 

b.  When  to  measure  1  1 

c.  How  to  measure  2  2 

d.  What  training 
devices  are  used 

to  measure  1  5 


6 

7 

4 


1 

1 

2 


2  1 


It  was  important  to  determine  exactly  which  PMA  information 
was  important  and  necessary  for  the  KDMs  to  make  their  decisions. 
Table  2  shows  the  respondents'  reactions  to  the  need  for,  and 
availability  of,  certain  types  of  PMA  data.  Table  3  displays  the 
respondents'  self-generated  responses  to  an  open-ended  question 
about  the  additional  types  of  PMA  data  which  were  needed  by  the 
KDMS. 
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TABLE  2 


Question  3a.  What  kind  of  information  does  the  KDM  need/get  to 


make  decisions? 

Need 

this 

data? 

Get 

this 

data? 

Quantitative 
or  Oualitativi 

• 

YES 

NO 

YES 

NO 

QUAL 

QUAN 

1.  Feedback  from 

Active  Units 

9 

0 

8 

1 

9 

5 

2.  Training 

Effectiveness 

Data  (from  the 
Institution) 

9 

0 

7 

2 

6 

6 

3 .  Feedback  from 
Instructors 

9 

0 

7 

2 

7 

1 

4.  Training  Device 
Effectiveness 

Data 

4 

3 

3 

5 

3 

3 

5.  Cost  Effec¬ 
tiveness  Data 

4 

4 

1 

7 

0 

1 

6.  Dept,  of  Evalu¬ 
ation  and 
Standardization 

8 

1 

6 

1 

7 

6 

7.  End  of  Cycle 

Test  Results  (from 
the  Institution) 

8 

1 

8 

1 

2 

8 

8.  Test  Validity 
and  Reliability 
Data 

7 

2 

5 

3 

4 

3 

9.  Knowledge  of 
Learning  Rates 

6 

2 

1 

7 

1 

1 

10.  Skill  and 

Qualification 

Test  Results 

7 

1 

7 

1 

4 

7 

*  Where  responses  exceed  the  number  of  respondents  (9) , 
some  respondents  indicated  that  they  received  both 
quantitative  and  qualitative  types  of  data. 
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TABLE  3. 


>e  of  PMA  data  rectuired/received  f open-ended  auestlonl 


Question  3b.  If  the  KDM  needs  other  information  not  listed 
above,  then  list  other  types  of  data  and  indicate  if  the  KDM  gets 
this  data  by  placing  a  check  mark  under  the  "YES”  or  "NO"  column. 


Other  data  needed 


GETS  THIS  DATA 
YES  NO 


1.  Media  Effectiveness  X 

2.  Skill  Perishability  X 

3 .  Resource  Constraints 
(e.g.,  vehicle,  land, 
fuel,  ammo,  spare  parts 

availability)  X 

4 .  Feedback  from  Reserve  Units  X 

5.  Field  Returnee  Reports  X 

6.  Learning  Decay  X 

7.  Training  Validation  Data  X 

8.  Job  Performance  Data  X 
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Table  4  displays  the  respondents  estimations  about  which 
sources  of  PMA  data  are  most  important.  The  average  rank  and 
standard  deviation  for  each  source  are  calculated. 


TABLE  4 .  Most  important  sources  of  PMA  data . 


Question  4.  Please  indicate  how  important  you  believe  the  data 
selected  for  questions  3a.  and  3b.  are  when  decisions  are  made  about 
training  measurement  and  assessment.  First,  SELECT  10  types  of  data 
you  believe  are  most  important  when  making  decisions.  Second,  RANK 
ORDER  your  selections.  Let  1  indicate  "very  important”  and  10 
indicate  "least  important”  of  those  selected. 


Average 

Selection  Rank 

End  of  Cycle  Test  Results  1.7 

(From  the  Institution) 

Feedback  from  Active  Units  2.2 

Department  of  Evaluation 

and  Standardization  Data  2.3 

Test  Validity/Reliability  3.2 

Data 

Feedback  from  Instructors  3.8 

Training  Device  Effectiveness 

Data  4 . 0 

Skill  and  Qualification  Test 

Results  4.4 

Knowledge  of  Learning  Rates  5.0 

Cost  Effectiveness  Data  5.3 

Training  Effectiveness  Data  5.8 

(From  the  Institution) 


Standard 

Deviation 

.8 


1.9 

1.0 

1.3 

2.2 

1.4 

2.1 

2.0 

2.6 

2.8 


In  addition  to  the  selections  shown  above,  some  respondents 
ranked  additional  items  (see  3b.  above) .  These  items  are  not 
included  in  the  table  above  because  each  received  only  one 
response. 
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The  second  questionnaire  presented  to  the  KDMs  (DOTD) 
consensus  statements  about  the  results  of  the  first 
questionnaire.  The  intent  was  to  check  the  reliability  of  the 
findings  from  the  first  questionnaire  and  to  allow  the  KDMs  to 
address  additional  areas  of  PMA  when  they  so  wished.  Tables  5 
through  11  display  responses  to  each  of  the  six  questions. 


TABLE  5 .  Key  decision  makers  at  the  school . 

Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NO 

Question  1.  Who  at  the  school  makes 
decisions  on  training  measurement  and 

assessment?  DOTD/DOES  9  2 

Rationale  of  those  who  did  not  agree  with  the  consensus; 

1.  Under  school  Model  83,  the  training  departments  take  the 
lead  for  design,  development,  and  implementation. 

Therefore,  many  "measurement”  action  are  theirs. 

2.  DOTD/DOES  play  a  major  role,  however  do  not  forget  the 
AC/CG  input.  They  drive  the  train. 


TABLE  6 .  Impact  of  performance  measurement  and  assessment 
decisions. 


Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NO 

Question  2a.  How  often  do  your 
decisions  affect  WHAT  performance  is 

measured?  Generally  9  2 

Rationale  of  those  who  did  not  agree  with  the  consensus; 

1.  Again,  it  goes  back  to  the  Commanding  General. 

2.  "What"  is  a  subject  matter  decision  and  is  determined  by 
the  school  director. 

Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NO 

Question  2b.  How  often  do  your 
decisions  affect  WHEN  to  measure 

performance?  Generally  10  1 
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TABLE  6 .  Impact  of  performance  measurement  and  assessment 
decisions  (continued) . 


Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NO 

Question  2c.  How  often  do  your 
decisions  affect  HOW  to  measure 

performance?  Generally  10  1 

Rationale  of  the  respondent  who  did  not  agree  with  the  consensus: 

1.  The  subject  matter  expert  has  the  greatest  measurement 
input. 


Consensus  Do  you  agree 

from  first  with  Consensus? 

Questionnaire 

YES  NO 

Question  2d.  How  often  do  your 
decisions  affect  what  TRAINING  DEVICES 

are  used  to  measure  performance?  Sometimes  7  4 

Rationales  of  those  who  did  not  agree  with  the  consensus: 

1.  Although  the  answer  is  presently  "sometimes,”  it  will  soon 
change  to  "nearly  always"  as  the  emphasis  on  training 
devices  increases. 

2.  No  one  examines  an  alternate  answer  to  training  devices, 
such  as  computer-based  instruction. 

3.  Probably  more  frequently  than  "sometimes." 
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TABLE  7. 


PMA  Information  required  for  decision  making. 

Consensus  Do  you  agree 
from  first  with  Consensus? 
Questionnaire 

YES  NO 

Question  3a.*  What  kind  of  information 

do  you  need  to  make  your  decisions?  Feedback 

from  Units  8  3 

Rationale  of  those  who  did  not  agree  with  the  consensus; 

1.  Too  much  valuable  time  is  lost  evaluating  and  replying  to 
field  suggestions  that  are  clearly  worthless. 

2.  The  units  have  a  very  narrow  view.  They  are  only  concerned 
with  what's  best  for  their  specific  units. 

3.  No  two  units  are  managed  the  same,  therefore  doctrine  is 
seldom  implemented  as  intended.  Feedback  data  in  99.99%  of 
the  cases  is  a  compilation  of  opinions  in  different 
environments . 


Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NQ 

Question  3a.  What  kind  of  information  Training 

do  you  need  to  make  your  decision?  Effectiveness  10  1 

Data 

Rationale  from  the  respondent  who  did  not  agree  with  the  consensus: 

1.  Such  feedback  is  just  not  included  in  the  instruction  or 
evaluation  at  our  school. 

Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NQ 

Question  3a.  What  kind  of  information  Feedback 
do  you  need  to  make  your  decisions?  from 

Students  10  1 


*  NOTE;  Question  3a  was  repeated  three  times  because  the 
results  of  the  first  questionnaire  showed  these 
three  types  of  information  to  be  highly  rated 
by  the  respondents. 
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TABLE  8. 


)ualitative  vs.  quantitative  PMA. 


Question  4.  Is  the  information 
qualitative  or  quantitative? 


Consensus  Do  you  agree 
from  first  with  Consensus? 
)uestionnaire 

YES  NO 
Qualitative  7  2 


Rationale  of  those  who  did  not  agree  with  the  consensus: 

1.  We  use  both  types  extensively. 

2.  Although  we  presently  use  primarily  qualitative  data,  we 
are  attempting  to  use  more  quantitative  sources  to  meet  the 
goal  of  presenting  our  case  more  effectively  to  the 
Department  of  the  Army  and  Congress. 


TABLE  9.  Most  important  PMA  data. 

Consensus  Do  you  agree 

from  first  with  Consensus? 
Questionnaire 

YES  NO 


Question  5.  What  type  of  data  is  Feedback 

most  important  when  making  decisions  from  active 
on  training  and  assessment?  units  and 

end-of-cycle  tests 
in  the  institutions 


Rationales  of  those  who  did  not  agree  with  the  consensus: 

1.  End-of-cycle  test  yes,  but  feedback  from  active  units  is  in 
some  cases  diametrically  opposed  to  data  from  other  units. 

2.  We  must  include  the  Reserve  Component  in  our  decision 
process.  They  are  of  primary  importance. 

3 .  Our  two  most  important  sources  of  information  are  the  units 
and  feedback  from  our  (the  school's)  field  units. 
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TABLE  10.  Least  important  PMA  data. 

Consensus  do  you  agree 
from  first  with  Consensus? 

Questionnaire 

YES  NO 

Question  6.  What  type  of  data  is  least  Training 

important  when  making  decisions  on  Effectiveness 

training  measurement  and  assessment?  Data  6  3 

Rationale  of  those  who  did  not  agree  with  the  consensus: 

1.  I  believe  training  effectiveness  data  is  most  important, 
followed  by  unit  feedback. 

2.  Our  least  effective  source  is  the  students  while  attending 
the  course.  They  are  just  too  focused  on  "how  well"  they 
are  doing  vs.  any  objective  look  at  their  performance 
against  a  criterion. 


Discussion 

Although  this  portion  of  the  study  did  not  derive  data  from 
every  Key  Decision  Maker  (KDM)  in  the  Army  schools,  it  did  access 
information  from  a  good  percentage  (39%  of  the  schools  for  the 
first  questionnaire  and  48%  for  the  second) .  It  explored  an  area 
of  training  PMA  which  has  apparently  not  been  previously 
examined.  It  allowed  the  researchers  to  gain  some  understanding 
about  PMA  information  requirements  and  uses.  Perhaps  most 
importantly  it  revealed  a  number  of  PMA  issues  which  should  be 
more  closely  examined  in  future  research. 

In  general,  the  results  of  the  second  questionnaire  validated 
the  first.  There  are  two  possible  exceptions  to  that  statement. 
The  first  questionnaire  revealed  that  the  KDMs  did  not  view 
information  about  training  device  effectiveness  as  being 
particularly  important.  However,  certain  responses  on  the  second 
instrument  gave  indication  that  a  number  of  the  KDMs  are, 
beginning  at  least,  to  value  the  PMA  information  which  can  be 
provided  by  the  training  device. 

One  of  the  problems  with  past  and  present  training  devices 
has  been  their  confusing  and  complicated  PMA  systems,  for  those 
devices  which  have  such  systems.  The  instructors  are  usually 
inundated  with  PMA  data  since  digitally  based  devices  can  record 
every  possible  trainee  and  instructor  action.  This  "ocean"  of 
data  usually  overwhelms  the  instructor  and  is  often  not  used  for 
that  reason.  As  this  problem  becomes  better  understood,  PMA 
systems  are  beginning  to  be  designed  more  often  with  the  user  and 
KDMs  in  mind.  It  is  perhaps  for  this  reason  that  the  KDMs  who 
responded  to  the  second  questionnaire  showed  more  interest  in 
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Training  Device  Effectiveness  data  than  was  evident  in  the  first 
response.  It  is  important  to  note  here  that  the  set  of 
respondents  for  the  second  instrument  was  net  identical  to  the 
set  that  responded  to  the  first  questionnaire. 

The  second  questionnaire  results  were  less  unanimous  about 
the  importance  of  Feedback  from  Active  Units  as  a  PMA  data 
source.  There  were  indications  that  some  KDMs  felt  the  units 
were  too  narrowly  focused  on  the  units'  specific  concerns  to 
provide  valid  feedback  data.  In  other  words,  such  data  may  be  so 
biased  that  it  is  of  little  use  to  a  KDM  who  must  train 
individuals  for  all  of  the  Army.  It  is  possible  that  the  data 
might  be  more  useful  if  the  units  received  instruction  about  the 
PMA  needs  of  the  schools  and  were  given  structured 
interviews/questionnaires  that  specifically  addressed  the 
feedback  needs  of  the  school. 

The  survey  revealed  that  the  KDMs  need  information  about 
Cost-Effectiveness  Data  and  Trainee  Learning  Rates.  However,  it 
is  clear  that  the  KDMs  do  not  receive  this  information  on  a 
regular  basis.  Cost-effectiveness  data  can  help  KDMs  to 
determine  whether  the  Army  is  getting  the  best  training  outcome 
for  its  training  resource  investment.  It  can  also  aid  in 
obtaining  additional  resources,  if  required,  by  quantitatively 
showing  potential  sponsors  quantitative  areas  of  strength  and 
weakness  in  the  training  program. 

However,  such  cost-effectiveness  data  is  obviously  difficult 
to  obtain.  It  requires  a  firm,  quantitw.:ive  understanding  of 
both  the  resource  input  and  the  learning  outcome  of  a  training 
program.  Quantifying  resource  inputs  is  difficult  in  the 
military  because  many  costs  are  indirectly  charged  to  the 
training  program.  For  example,  while  we  can  calculate  the  cost 
of  training  an  instructor  for  school  duty,  it  is  nearly 
impossible  to  determine  the  lost  cost  benefit  of  not  having  that 
instructor  in  a  combat  unit. 

In  like  manner,  it  is  often  not  posc'’''''.?  Ic  q’-^r.titatively 
determine  the  learning  outcome  of  a  training  program.  Our 
measurement  techniques  are  still  developing  and  it  is  clear,  as 
evidenced  by  the  responses  to  the  questionnaires,  that  KDMs  are 
not  receiving  enough  of  this  information.  The  end  result  is  a 
general  quandary  about  the  cost  effectiveness  of  training 
programs.  Managers  would  like  this  PMA  data  but  it  is  not 
forthcoming. 

Data  about  the  learning  rates  of  trainees  can  help  KDMs 
answer  such  questions  as: 

•  Is  this  group  of  trainees  learning  as  quickly  as 
previous  trainees? 

•  Based  upon  their  learning  rate  can  we  cut  the  length  of 
a  course?  Do  we  need  to  lengthen  the  course? 
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•  Should  remediation  sessions  be  made  available  to 

trainees  based  upon  their  slower  than  average  learning 
rates? 

To  the  degree  that  quantitative  learning  data  is  not  provided 
to  KDMs  they  will  have  to  rely  mainly  on  instructor  feedback. 
Certainly  such  feedback  is  valuable,  but  it  is  always  prone  to 
bias  and  thus  becomes  less  valuable  than  if  it  were  paired  with 
quantitative  learning  rate  data. 

The  difficulty  in  measuring  training  outcomes,  as  mentioned 
above,  is  largely  responsible  for  the  seemingly  contradictory 
finding  about  the  need  for  Training  Effectiveness  Data.  When 
asked  about  their  need  for  such  data,  the  KDMs  unanimously 
responded  that  it  was  needed.  However,  when  they  were  asked  to 
rank  order  the  importance  of  various  PMA  sources.  Training 
Effectiveness  Data  was  ranked  last.  This  discrepancy  is  likely 
due,  again,  to  the  difficulty  in  obtaining  valid  Training 
Effectiveness  Data.  For  example,  the  survey  showed  that  79%  of 
the  respondents  received  this  type  of  data  and  yet  they  still 
ranked  it  as  least  important  of  all  the  PMA  sources.  Such  a 
finding  can  be  interpreted  as  a  non-vote  of  confidence  for  the 
efficacy  of  the  Training  Effectiveness  Data  which  is  presently 
being  generated. 

Another  seeming  anomalous  finding,  concerns  the  importance  of 
Feedback  from  Instructors  as  a  PMA  data  source.  The  KDMs 
unanimously  stated  that  such  feedback  is  needed.  Yet,  such  data 
was  only  ranked  fifth  out  of  ten  when  compared  to  other  PMA 
sources.  This  may  be  yet  another  indication  that  KDMs  value 
instructor  feedback  but  in  absence  of  quantitative  Training 
Effectiveness  Data  as  a  cross-check,  the  KDMs  tend  to  often  look 
elsewhere  for  valid  PMA  information. 

One  final  area  of  discussion.  The  1985  Army  Science  Board 
made  a  strong  point  about  the  need  for  quantitative  PMA  data. 

The  concept  is  that  significant  increases  in  training 
effectiveness  can  only  come  about  as  we  are  better  able  to 
measure  where  the  training  systems  have  come  from,  where  they  are 
now,  and  where  they  are  going.  The  findings  of  this  survey  tend 
to  support  the  ASB's  recommendation  that  more  quantitative  data 
is  required.  The  ASB's  contention  that  quantitative  data  is 
generally  unavailable  is  not  supported  however  by  one  finding. 

The  KDMs  indicated  that  the  PMA  data  they  have  access  to  is  at 
least  as  much  quantitative  in  nature  as  it  is  qualitative.  The 
one  exception  is  in  the  area  of  Feedback  from  Instructors,  which 
was  discussed  above  as  being  a  problematic  data  source. 

It  is  entirely  possible  that  the  KDMs  and  the  ASB  are 
defining  qualitative  and  quantitative  data  differently.  It  is 
not  clear  in  the  ASB  study  how  the  Board  defines  the  terms.  The 
researchers  in  this  study  did  not  ask  the  respondents  for  their 
definitions,  although  such  additional  information  would  be 
helpful.  Despite  these  definitional  issues,  it  appears  that  the 
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true  amount  and  type  of  quantitative  PMA  data  which  exists  in  the 
Army's  training  systems  should  be  a  continuing  source  of  study. 

Rpcommendations 


This  brief  survey  has  shovm  that  KDMs  in  Army  schools  do  not, 
in  many  cases,  receive  the  type  of  PMA  data  they  feel  they 
require.  Future  research  should  continue  to  define  the  types  of 
data  required.  Effective  methods  should  be  developed  for 
gathering  such  data.  This  survey  topic  should  also  be  examined 
with  other  KDMs  not  directly  associated  with  the  daily  operation 
of  the  schools  (e.g.,  Training  Doctrine  Command  (TRADOC) ,  Deputy 
Chief  of  Staff  for  Operations  and  Plans  (DCSOPS) ,  Deputy  Chief  of 
Staff  for  Personnel  (DCSPER) ,  US  Forces  General  Command  (FORSCOM, 
US  Army  Europe  (7th  Army)  (USAREUR) ,  Department  of  Army  (DA)  , 
Department  of  Defense  (DoD,  and  Congress) .  Each  of  these 
agencies  and  organizations  has  their  own  PMA  information  needs, 
but  the  literature  this  study  has  examined  has  not  revealed  any 
information  about  those  needs.  Detailed  techniques  such  as 
Policy  Capturing  and  Policy  Implications  Analysis  could  help 
those  interested  in  improving  Army  training  to  describe  the 
present  PMA  information  needs  and  uses  of  and  by  school  KDMs. 

They  also  could  help  predict  what  data  would  be  of  most  use.  In 
turn,  those  techniques  applied  to  non-school  organizations  would 
predict  their  future  PMA  information  needs.  As  our  technologies 
and  capability  for  gathering  PMA  data  increase,  such  information 
about  KDM  needs  is  vital. 
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SUMMARY 


There  is  much  good  in  the  Army's  way  of  training  and  measuring 
performance.  Although  there  are  undoubtedly  perturbations  in  the 
system,  performance  is  generally  derived  from  the  unit's  mission 
and  the  job  requirements  of  the  individual  soldiers.  For  the 
most  part,  the  training  PMA  is  based  on  job  or  mission  related 
criteria  and  is  performance-oriented.  In  addition,  training  PMA 
data  are  used  to  make  decisions  on  additional  training  required 
in  all  the  various  training  environments.  Finally,  there  is  an 
awareness  of  the  importance  of,  if  not  a  method  of  accomplish¬ 
ment,  for  making  performance  standards  as  objective  as  possible. 
If  the  personnel  who  were  interviewed  as  part  of  this  survey  are 
in  fact  a  representative  sample  of  the  individuals  who  develop, 
conduct  and  supervise  the  Army's  training,  then  the  Army's  train¬ 
ing  system  is  in  the  hands  of  some  very  competent  and  dedicated 
people. 

On  the  negative  side,  there  appears  to  have  been  little 
progress  made  in  remedying  the  problems  noted  during  the  Army 
Science  Board's  1985  Summer  Study.  In  very  general  terms,  the 
authors  have  concluded  that  the  Army  has  achieved  little,  if  any, 
success  at  developing  an  integrated  and  cohesive  system  for 
measuring  training  performance  and  using  the  results  to  improve 
the  training  of  its  individuals  and  units.  Obviously  there  are 
probably  exceptions  to  that  generalization;  no  conclusive 
evidence,  however,  was  observed  to  suggest  that  the  assumption  is 
not  valid. 

There  were  several  specific  problems  revealed  during  the 
study  effort.  For  instance,  there  is  still  a  great  deal  of 
reliance  on  subjective  measures  of  performance.  Evaluators  are 
called  on  to  measure  performance  against  standards  which  require 
the  evaluator  to  make  judgment  calls  on  the  adequacy  of  the 
performance.  This  appears  to  be  especially  true  in  the  area  of 
collective  training,  although  there  are  similar  instances  in  the 
individual  training  environment. 

The  validity  and  reliability  of  the  training  PMA  system  is 
likewise  open  to  criticism.  Philosophically,  the  Army  seems  not 
to  have  made  a  decision  as  to  how  best  to  evaluate  performance. 

A  lingering  problem  is  whether  performance  should  be  measured  in 
a  rigid,  tightly  controlled  and  scenario-driven  environment, 
thereby  achieving  a  higher  degree  of  reliability  while 
sacrificing  some  realism,  or  whether  performance  should  be 
measured  in  an  unstructured  or  non-standardized  manner,  thus 
increasing  realism  while  sacrificing  reliability.  Moreover, 
there  is  evidence  to  suggest  that  mechanisms  which  would  optimize 
the  validity  of  the  training  PMA  methodologies  either  are  not 
available,  are  misunderstood  by  trainers  and  training  developers, 
or  are  not  being  used. 
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The  question  of  process  versus  product  orientation  as  it 
applies  to  training  PMA  is  a  perplexing  one.  While  a  process 
orientation  is  important  in  initial  training,  and  is  arguably 
important  in  instances  where  the  training  PMA  system  is  being 
used  to  provide  a  diagnostic  function,  it  may  be  less  than  ideal 
in  other  circumstances.  Since  training  PMA  systems  that  are 
process-oriented  are  by  their  nature  costly  in  terms  of  evaluator 
time,  they  tend  to  be  inefficient.  This  leads  inevitably  to  the 
conclusion  that  the  training  PMA  system  used  within  the  Army 
could  be  made  more  efficient,  if  not  more  effective,  if 
product-orientation  methodologies  were  applied  in  the  appropriate 
settings. 

Finally,  there  seems  to  be  a  problem  with  communication  in 
the  training  and  training  PMA  arenas.  To  a  certain  extent,  this 
may  be  the  result  of  the  plethora  of  publications  to  which 
trainers,  training  managers,  and  training  developers  must  refer. 
For  instance,  trainers  and  training  managers  in  a  unit  are  faced 
with  soldier's  manuals,  trainer's  guides,  job  books,  field 
manuals,  technical  manuals,  unit  Standard  Operating  Procedures 
(SOPs) ,  and  ARTEPs,  all  of  which  may  contain  information 
pertaining  to  individual  and  collective  tasks  and/or  standards. 
Compounding  the  problem  is  the  variety  of  training  terms,  many  of 
them  contradictory,  contained  in  Army  training  publications. 

There  is  abundant  evidence  that  terms  such  as  quantitative  versus 
qualitative  training  PMA,  individual  versus  collective  tasks,  and 
the  variety  of  terms  used  to  categorize  training  (individual, 
collective,  institutional,  unit,  integration,  initial  entry, 
sustainment,  refresher,  train-up,  cross-training,  and 
enhancement)  are  confusing  to  those  who  develop  and  implement 
training  throughout  the  Army.  In  order  for  training  and  training 
PMA  to  improve  significantly,  the  Army  needs  to  insure  that  the 
terminology  used  is  understood  by  all  who  are  involved. 

Future  research  efforts  should  be  pursued  to  help  alleviate 
the  problems  mentioned  above.  Some  of  these  efforts  might 
include:  studies  to  determine  the  potential  benefits  of  greater 

utilization  of  a  product-oriented  performance  measurement  system; 
studies  on  the  potential  advantages  of  using  quantitative 
measures  of  performance  in  lieu  of  qualitative  measures;  methods 
for  providing  key  decision  makers  necessary  training  PMA 
information;  and  studies  to  determine  methods  for  improving  the 
flow  of  performance  measurement  data  between  the  schools  and 
units.  Each  of  these  study  efforts  would  provide  the  Army  with 
insights  into  how  to  proceed  to  overcome  problems  surfaced  during 
previous  studies,  such  as  the  Army  Science  Board's  Summer  Study 
of  1985.  There  are  problems  which  this  effort  found  are  still 
unresolved. 
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ACRONYMS 


ARI 

APM 

AC 

AFHRL 

ATB 

ARTEP 

AI 

ASB-1985 

BTIS 

COTEAM 

CATA 

CO 

OTIC 

DOES 

DOTO 

DA 

DoD 

DCSOPS 

DCS PER 
ERIC 
FORSCOM 
GAO 


Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences 

Advanced  Simulator  for  Pilot  Training 
(APM) 

Active  Component 

Air  Force  Human  Resource  Laboratory 
Army  Training  Board 

Army  Training  and  Evaluation  Program 

Artificial  Intelligence 

Army  Science  Board  1985  Summer  Study  on 
Training  and  Training  Technology 

Branch  Training  Teams 

Combat  Operations  Training  Effectiveness 
Model 

Combined  Army  Training  Activity 
Commanding  Officer 

Defense  Technical  Information  Center 

Department  of  Evaluation  and 
Standardization 

Directorate  of  Training  and  Doctrine 
Department  of  Army 
Department  of  Defense 

Deputy  Chief  of  Staff  for  Operations  and 
Plans  (HQDA) 

Deputy  Chief  of  Staff  for  Personnel  (HQDA) 
Educational  Resources  Information  Center 
US  Army  Forces  Command 
General  Accounting  Office 
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HQTRADOC 

KDM 

MOS 

MILES 

MTP 

NCO 

NETT 

OSUT 

ODCST 

POI 

PMS 

PMTRADE 

RC 

STP 

SMCT 

SQT 

TM 

TO&E 

TRADOC 

TPDC 

TRASANA 

TEA 

T&EOs 

USAREUR 


ACRONYMS  (continued) 

Headquarters,  Training  and  Doctrine 
Command 

Key  Decision  Maker 

Military  Occupational  Specialty 

Multiple  Integrated  Laser  Engagement 
System 

Military  Training  Plan 

Non  Commissioned  Officer 

New  Equipment  Training  Team 

One-Station-Unit-Training 

Office  of  the  Deputy  Chief  of  Staff  for 
Training 

Program  of  Instruction 

Performance  Measurement  and  Assessment 

Project  Manager  for  Training  Devices 

Reserve  Component 

Soldier  Training  Product 

Soldier  Manual-Common  Tasks 

Skill  Qualification  Test 

Technical  Manual 

Table  of  Organization  and  Equipment 
US  Army  Training  and  Doctrine  Command 
Training  and  Performance  Data  Center 
TRADOC  Systems  Analysis  Agency 
Training  Effectiveness  Analysis 
Training  and  Evaluation  Outlines 
US  Army  Europe  (7th  Army) 


34 


GLOSSARY 

Term  Definition 

Active  Component  The  full-time  component,  on  active 

duty,  of  the  Total  Force 

Basic  Combat  Training  Basic  introductory  and 

indoctrination  training  given  to 
newly  inducted  enlisted  Active  and 
Reserve  personnel  without  prior 
military  service.  It  provides  a 
transition  from  civilian  to 
military  life,  motivation  to 
become  a  dedicated  and  productive 
member  of  the  Army,  and 
instruction  in  the  basic  skills, 
to  include  battlefield  survivial 
skills,  required  by  all  members  of 
the  Army.  (TRADOC  Cir  350-3) . 

Collective  Training  Training,  either  in  institutions 

or  units  that  prepares  a  group  of 
individuals  (crews,  teams,  squads, 
platoons,  etc.)  to  accomplish 
tasks  required  of  the  group  as  an 
entity  (TRADOC  Cir  350-3) 

Criterion  Referenced  Test  A  test  which  measures  what  an 

individual  must  be  able  to  do  or 
must  know,  in  order  to 
successfully  perform  a  task.  An 
individuals'  test  performance  is 
compared  to  an  external 
criterion/prespecified  performance 
standard  which  is  derived  from  an 
analysis  of  what  is  required  to  do 
a  particular  task  (TRADOC  Cir 
350-3) . 

Enhancement  Training  A  group  of  training  situations 

wherein  a  a  soldier  in  a  unit  is 
trained  to  perform  tasks  for  which 
he/she  has  had  previous  training. 
Enhancement  training  may  include 
cross-training,  new  equipment 
training  or  train-up  training. 

Individual  Training  Training  which  the  individual 

officer,  NCO,  or  enlisted  person 
receives  in  institutions,  units  or 
by  extension  self-study,  that 
prepares  the  individual  to  perform 
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GLOSSARY  (continued) 


Term 


Definition 


Ir.sti't'ut ionai  Training 


Integration  Training 


Job  Book 


Norm-Referenced  Test 


Objectivity 


specified  duties  and  tasks  related 
to  the  assigned  MOS  and  duty 
position  (TRADOC  Cir  350-3) . 

Training,  either  individual  or 
collective,  conducted  in  schools 
(Army,  service  school,  USAR 
school,  NCO  Academy,  unit  school) 
or  Army  Training  Centers 
(TRADOC  Cir  350-3). 

Training  that  a  soldier,  newly 
arrived  in  a  unit  receives  to 
complete  his  Initial  Entry 
training  on  skill  level  1  tasks 
(BTMS -AC-8 3-1)  . 

A  book  for  the  NCO  supervisor 
useful  in  maintaining  empirical 
data  (training  record)  on  a 
soldier's  work  performance  in  his 
duty  position.  It  assists  the  NCO 
supervisor  in  keeping  track  of  a 
soldier's  ability  to  perform  the 
tasks  of  a  duty  position,  and 
planning  to  meet  the  training 
needs  of  the  individual  soldier 
(TRADOC  Cir  350-3) . 

A  test  which  grades  a  student  in 
relation  to  the  performance  of 
other  students  in  contrast  to 
criterion-referenced  testing 
wherein  a  student  is  graded  in 
relation  to  a  prespecified 
performance  standard 
(TRADOC  Cir  350-3). 

In  testing,  the  degree  to  which  a 
test  is  scored  the  same  by  two  or 
more  scorers  acting  independently 
(TRADOC  Cir  350-3). 


One-Station-Unit-Training  Initial  entry  training  conducted 

at  one  installation  in  one  unit 
with  the  same  cadre  and  one  POI. 

BT  and  AIT  instruction  are 
integrated  in  order  to  permit  the 
early  introduction  of  MOS  specific 
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GLOSSARY  (continued) 


Term  Definition 

training,  followed  by  adequate 
reinforcement  training  to  assure 
mastery  (TRADOC  Cir  350-3). 

Performance  Assessment  The  instructor  synthesizes  all 

*  performance  meacureme’^t 

information  to  assess  trainee 
performance.  The  performance 
measures  may  be  objective  (e.g., 
machine  generated  information  such 
as  number  of  target  hits)  or 
subjective  (e.g.,  information 
gathered  through  the  instructor 
senses  as  proper  communication 
format  used)  (MIL-HDBK-220B) . 

Performance  Measurement  The  scoring  of  trainee  proficiency 

either  subjectively  (e.g. , 
instructor  opinion)  or  objectively 
(e.g.,  automatic  computer 
measurement)  (MIL-HDBK-220B) . 

Process  Task  A  task  which  consists  of  a  series 

of  steps  resulting  in  the  soldier 
obtaining  a  single  discrete 
result.  The  task  is  evaluated  by 
observing  the  process  and  by 
scoring  each  step  or  element  as  it 
is  performed  in  terms  of  sequence, 
completeness,  accuracy,  or  speed. 
Examples  are  "put  on  the 
protective  mask”  and  "take  oral 
temperature.”  (TRADOC  Cir  350-3). 

Product  Task  A  task  which  terminates  in  a 

discrete  product  or  outcome  which 
is  observable  and  measureable. 

The  task  is  evaluated  by  looking 
at  the  product  or  outcome  in  terms 
of  completeness,  accuracy, 
tolerance,  clarity,  error  or 
quantity.  "Repair  the  carburetor" 
could  also  be  an  example  of  a 
product  task  (TRADOC  Cir  350-3) . 

Qualitative  A  term  describing  a  performance 

measurement  standard  that  relies 
on  objective  ratings  or  word 
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GLOSSARY  (continued) 


Term 


Quantitative 


Reliability 


Reserve  Component 


Soldier's  Manual 


Sustainment  Training 


Training  and  Evaluation 
Plan 


Definition 

descriptions  to  determine  adequacy 
of  performance. 

A  term  describing  a  performance 
measurement  standard  that  relies 
on  numbers  to  determine  adequacy 
of  performance. 

The  degree  to  which  a  test 
instrument  can  be  relied  upon  to 
yield  the  same  result  upon 
repeated  administrations  to  the 
same  population  (TRADOC  Cir 
350-3)  . 

The  part-time  portion  of  the  Total 
Forces  e.g.,  the  Army  Reserve  and 
Army  National  Guard. 

A  manual  that  lists  for  the 
soldier  those  critical  tasks 
needed  to  perform  satisfactorily 
at  his  present  skill  level.  In 
addition,  the  SM  tells  the  soldier 
how  to  perform  the  tasks,  the 
expected  conditions  under  which 
they  will  be  performed,  and  the 
standards  which  must  be  met.  The 
SM  is  the  basis  for  the  tasks  used 
in  the  SQT  (TRADOC  Cir  350-3). 

Training  a  soldier  receives  in 
order  for  him/her  to  maintain 
proficiency  in  a  skill  previously 
learned. Technical  ManualA 
publication  containing  a 
description  of  the  equipment, 
weapons,  or  weapons  systems  with 
instructions  for  effective  use, 
including  one  or  more  of  the 
following  sections,  as  required: 
maintenance  instructions, 
modification  instructions,  repair 
parts  lists,  etc. 

(PM  TRADE  Manual) . 

A  document  prepared  by  a  TRADOC 
school  for  each  resident  course 
that  prescribes  the  training  and 
testing  concept  of  each  course. 
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GLOSSARY  (continued) 


Term 

Trainers'  Guide 


Unit  Training 

Validity 


Definition 

A  publication  prepared  by  an  MOS 
proponent  that  specifies  tasks 
taught  to  standards  in  the 
training  base  and  those  that  must 
be  initially  trained  in  the  unit 
(intc7ri4tior.  training)  . 

Training,  individual  or 
collective,  conducted  in  a  unit 
(BTMSRC  83-1) . 

The  degree  to  which  a  test 
measures  what  it  purports  to 
measure  (Handbook  in 
Research  and  Evaluation) . 
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PERFORMANCE  MEASUREMENT  QUESTIONNAIRE  -  MOS  63NI0 


Respondent's  Position 


Experience. 


Date  of  Interview _  Piece  of  Interview _ 

Interviewer _ ^Tt«e  started _ ended 


1  Ever/  ccMjrte  or  tralrl.oj  progrcm  Is  founded  on  some  form  of  training  slrateg/.  it 
being  defined  as  a  statement  of  the  tasks  that  will  be  taught  to  the  soldiers,  where  the 
training  will  take  place,  how  It  will  be  conducted,  and  when?  Are  you  familiar  with  the 
strategy  on  which  the  training  program  for  MOS  63N 1 0  is  based? 


2. 


What  is  the  strategy?. 


rNT-j — ». 


§0  to  Ouostion  6 


3.  How  was  the  strategy  developed?  (Who  developed  it,  who  approved  it,  and  what 
influenced  the  development?) _ 


4.  m  your  opinion ,  can  the  strategy  be  changed  by  your  school? 

1- 


I  a.  Yes 

ULj_ 


b.  No 


^  §o  to  Ouostion  6 


5.  Under  what  circumstances  would  it  be?_ 


6.  AR  6 1 1  -20 1  contains  a  general  statement  of  the  tasks  that  a  soldier  with  MOS 
63N 1 0  performs  on  the  Job.  Here  is  that  list.  Are  you  familiar  with  which  of 
these  tasks  are  currently  being  taught  in  the  training  program? 


a.  Yes 


ygo  to  Ouostion  7 


b.No 


1 - ^  go  to  Ouostion  0 


A- 2 


7.  Pleese  Indicate  on  the  list  which  of  those  tasks  are  currenly  being  taught  at  the 
Armor  School  In  this  program. 


8.  Of  the  tasks  which  are  taught  In  the  training  program  for  MOS  63N 10.  what  Is  the 
procedure  for  developing  the  tasks,  conditions  and  standards? _ 


9.  A  particular  feedback  mechanism  -Performance  measurement  -  Is  designed  to  evaluate  the 
proficiency  of  a  soldier  In  accomplishing  certain  tasks;  It  Is  commonly  used  to  determine  If  a 
training  program  is  effective.  Are  you  familiar  with  how  performance  Is  measured  In  the 
training  proyam  for  MOS  o3N  1 0? 


1 0  Performance  can  be  measured  In  a  variety  of  ways,  such  as  end-of-cycle  tests,  observation 
of  class  participation,  quizzes,  graded  hands-on  ©cerclses.even  by  devices  which  measure 
performance  automatically.  Please  Indicate  with  a  check  mark  those  tasks  which  are  currently 
being  trained  In  this  program  which  are  subject  to  some  sort  of  performance  measurement.  In 
the  next  column.  Indicate  how  the  performance  Is  measured. 


1 1 .  If  devices  are  used  In  any  way  to  measure  po'formance  In  this  program ,  what  are  the 
devices  and  how  are  they  used?  In  other  words,  pleese  explain  the  process  used  to  measure 

performance  by  use  of  the  dev1ce(  s) _ 


1 2.  What  Is  the  purpose  of  performance  measurement  In  this  program?. 


1 3.  Who  determines  the  performance  measurement  methodologies  used  In  the  program?. 


1 A  Who  prepares  the  performance  measurement  instruments?. 


15.  How  are  Uw/ prepared?. 


1 6.  In  your  opinion,  Is  there  a  direct  relotionshlp  between  the  performance  that  is 
measured  In  the  program  and  the  performance  that  will  be  required  of  the  soldier  in  the 
job  environment? 

b.  No.  I 


1 7.  On  a  scale  of  1  to  5  ( 5  being  the  most  effective),  how  would  you  assess  the 
effectiveness  of  the  performance  measurement  system  being  used  in  this  training 
program? 


a.  Yes. 


Poor 


¥ery 


Effective 


I  I  I  I  I 

t  2  3  1  S 


18.  Reliability  Is  frequently  referred  to  as  the  measure  of  consistency;  In  the  context  of 
performance  measurement  Instruments.  It  is  the  measure  of  the  Instruments'conslstency  over 
time.  In  other  words,  con  the  Instruments  be  relied  on  to  accurately  measure  something  from 
one  course  to  the  next.  In  your  opinion,  on  a  scale  of  1  to  5-  (5  being  the  highest  ratlng)- 
are  the  Instruments  used  in  this  training  program  reliable? 


Poor 


¥ery 

Reliedle 


I  I  I  I  I 

3 


A- 4 


/ 


2 


4 


5 


19.  Validity  Is  frequently  referred  to  as  the  measure  of  appropriateness,  In  the  context 
of  the  performance  measurement  Instruments,  It  Is  the  measure  of  whether  the  Instrument 
does  In  fact  measure  what  It  claims  to  measure.  In  your  opinion,  on  a  scale  of  1  to  5  ( 5  being 
the  highest  rating),  are  the  Instruments  used  In  this  training  program  valid? 


Ysry 

Poor  Valid 

I _ I _ I _ I _ I 

I  2  3  4  S 


20.  Aside  from  the  questions  of  reliability  and  validity,  there  is  generally  an  opinion  oi 
perception  on  the  part  of  the  Instructors  and  students  of  the  accuracy  and  fairness  of  the 
performance  measurement  system.  If  you  are  familiar  with  how  the  overall  performance 
measurement  system  for  MOS  63N 1 0  is  regarded  by  the  instructors  and  students.  Indicate 
those  opinions  (using  I  for  Instructor,  S  for  Student)  on  the  following  scales: 


Inoccorate 

1 

1 

1 

1 

Very 

Accurate 

1 

/ 

2 

J 

d 

5 

Unfair 

1 

1 

1 

1  __ 

Vary 

Fair 

1 

/ 

2 

3 

d 

5 

2 1 .  Regardless  of  how  performance  is  measured,  some  form  of  measurement  results  are 
produced;  do  you  know  what  use  Is  made  of  the  performance  measurement  results? 


22.  Please  explain  how  the  results  are  put  to  use?. 


23.  Efficiency  can  be  described  as  production  of  something  effective  with  a  minimum  of  waste 
cost,  or  effort.  The  term  training  efficiency  relates  to  the  ability  to  produce  a  trained,  l.e. 
effective,  soldier  while  holding  down  the  resources  Involved  In  the  training  process  to  the 
minimum  practical.  Are  you  able,  from  your  perspective,  to  evaluate  how  efficient  this 
training  program  Is? 


a.  Yes 


b.  No  I - ►  go  to  Ouostion  28 


24.  What  Is  the  cost  ( ROM)  to  train  one  Individual  In  this  program? 


25.  What  cost  elements  are  used  to  figure  that  cost?. 


26.  In  your  opinion,  can  this  training  program  be  made  more  efficient?  l.e,  reduce  the  per 
student  costs? 


27.  How?. 


28.  Who  at  the  Armor  School ,  either  by  name  or  by  title.  Is  In  a  position  to  make 
decisions  as  to  what  should  or  should  not  be  done  with  respect  to  the  training  of  MOS  63N 1 0? 


29.  Which  of  the  following  decisions  do  they  make  or  have  Impact  on? 

a.  What  to  train?  _ 

b.  How  to  train?  _ 

c.  When  to  train?  _ 

d.  Where  to  train?  _ 


e.  How  to  evaluate? 


A- 6 


30.  In  your  opinion ,  In  order  for  them  to  make  such  decisions,  what  data  should  the 
decision  makers  be  receiving? - 


3 1 .  Again  In  your  opinion,  are  those  Individuals  getting  those  data? 

I  I  I  NO  I 

32.  If  It  were  in  your  power  to  change  the  course  of  Instruction  for  MOS  63N 1 0  ,  what 

would  you  do  to  Improve  It? - - 


33.  What  source  dxuments  should  we  look  at  in  order  to  gain  a  clearer  understanding  of 
how  MOS  63N 1 0  Is  being  trained  at  the  Armor  School? _ 


Name _ Unit _ Tel.  Nr. 

Position _ Experience _ 

How  are  your  platoon  Individual  training  requirements  determined? 


Who  makes  the  determination,  I.e.  who  decides  what  particular  individual  tasks  your 
soldiers  will  train  on  In  the  unit? 


What  Individual  tasks  are  soldiers  with  MOS _ required  to  perform  In  your 

unit? 


Are  those  tasks  oerformed  to  satisfy  training  requirements  or  to  satisfy  their  job 
requirements? 


What  document  lists  th®  <ndividual  tasks  soldiers  in  MC5 _ are  required  to 

perform? 


Who  prepares  the  training  materials  used  to  support  the  Individual  training 
conducted  in  your  platoon? 


How  do  you  know  the  conditions  under  which  these  tasks  should  be  trained? 


How  do  you  know  the  standards  to  which  each  task  should  be  performed? 


TRADOC  views  Individual  training  In  units  as  either  integration  training  (training  on 
Initial  entry  skills  when  the  soldier  first  arrives  in  the  unit),  sustainment 
training  (training  on  skills  previously  learned  either  In  the  school  or  the  unit), 
and  enhancement  training  (training  on  skills  not  previously  learned  but  which  are 
needed  for  advancement  or  for  perTorming  tasks  not  previously  performed,  either 
as  a  result  of  new  equipment,  new  doctrine,  etc.).  Do  you  break  down  Individual 
training  that  way  or  do  you  use  some  other  categorization? 


If  some  other  categorization  Is  used,  what  is  It? 
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What  tasks  do  you  teach  In  Integration  training?  How  do  you  conduct  that  training? 
How  do  you  measure  performance? 


What  tasks  do  you  teach  In  sustainment  training?  How  do  you  conduct  that 
training?  How  do  you  measure  performance? 


What  tasks  do  you  teach  In  enhancement  training?  How  do  you  conduct  that 
training?  How  do  you  measure  performance? 


What  percentage  of  your  training  time  do  you  spend  on  Individual  tasks? 


Is  there  a  means  for  measuring  the  performance  of  the  soldiers  on  these  individual 
tasks?  If  not  all,  approximately  what  percentage  of  the  tasks  are  subject  to 
performance  measurement? 


How  Is  Individual  task  performance  generally  measured? 


Who  prepares  the  performance  measurement  Instruments  you  use  to  evaluate  the 
training  of  Individual  and  collective  skills? 


When  you  measure  performance,  would  you  say  you  are  generally  more  Interested  In 
the  results  of  the  performance  or  the  process  involved  In  the  process?  [Give 
examples  If  It  will  help  the  respondent] 


On  a  scale  of  one  to  five  (five  being  the  highest  rating)  how  would  you  evaluate  the 
performance  measurement  system  you  use  to  measure  Individual  task 
performance  as  to: 

validity? _  Provide  examples  of  each 

reliability? _ category. 

accuracy? _ 

fairness? _ 

effectiveness? _ 

Do  you  use  a  pass/fall  measurement  system  or  a  graded  scale  system.  If  a  graded 
system,  Is  It  norm-based  or  criterion-based.  fUse  examples  to  explain  the 
difference  between  norm-based  and  criterion-based] 


Are  training  devices  ever  used  in  measuring  individual  task  performance?  If  so,  is 
the  decision  on  whether  the  standard  has  been  met  made  automatically  by  the 
device? 


If  the  pass/fail  or  grading  decision  is  made  by  a  human  observer,  is  the  decision 
objective  or  su^ective?  [Does  the  standard  itself  require  judgment  on  the  part 
of  the  observer?)  ^ 


What  criteria  do  you  use  to  determine  if  the  soldier  has  had  enough  training  on  a 
particular  task? 


Who  established  the  criteria? 


What  happens  if  a  soldier  continues  to  fail  to  reach  the  standard? 


Assuming  he  meets  the  appropriate  criteria,  how  do  you  determine  what  he  gets 
trained  on  next? 


Do  you  ever  amend  the  conditions  and/or  standards  that  are  prescribed  by  whatever 
source  document  you  use? 


Are  there  tasks  that  are  performed  by  your  soldiers  that  are  not  reflected  in  the 
source  document  that  ays  out  the  tasks  your  soldiers  should  perform? 


If  yes,  how  do  you  fit  those  into  your  training  schedule?  How  do  you  determine  the 
appropriate  conditions  and  standards? 


When  a  soldier  or  group  of  soldiers  completes  training  on  a  particular  individual 
task  and  has  his  or  their  performance  measured,  how  are  those  results  used? 


Are  there  any  individual  tasks,  critical  to  the  accomplishment  of  the  job  of  a 

soldier  with  MOS _ that  are  not  trained  in  the  school  prior  to  his  joining  the 

unit?  ^  ^ 
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If  yes,  what  are  those  tasks? 


Do  you  believe  that  these  tasks  should  be  taught  In  the  school? 


Do  you  know  why  those  tasks  are  not  trained  In  the  school? 


Are  there  any  Individual  tasks  that  soldiers  with  MOS _ are  trained  to  do  In  the 

school  but  which  they  cannot  perform  when  they  are  In  the  unit? 


Have  you  been  able  to  determine  why  they  are  unable  to  do  those  particular  tasks? 


Have  you  ever  talked  with  anyone  from  the _ School  about  the  ability  of 

their  graduates  to  perform  Individual  tasks  on  the  Job? 


Do  you  think  the _ School  Is  effectively  training  the  MOS _ soldiers  In 

performing  Individual  tasks? 


If  not,  what  should  they  do  differently? 


with  reference  to  collective  tasks,  how  are  your  platoon's  collective  training 
requirements  determined? 


Who  makes  the  determination,  i.e.  who  decides  what  particular  collective  tasks  your 
platoon  will  train  on? 


What  collective  tasks  are  soldiers  in  your  unit  with  MOS _ required  to 

perform? 


What  document  lists  the  collective  tasks  soldiers  In  your  unit  with  MOS _ are 

required  to  perform? 


Who  prepares  the  training  material  to  support  the  collective  training  requirements 
of  your  unit? 

How  do  you  know  what  conditions  these  tasks  should  be  trained  under? 

B-5 


How  do  you  know  what  standards  each  task  should  be  performed  to? 


TRADOC  views  collective  training  In  units  as  that  training  which  prepares  groups  of 
soldiers  to  perform  collective  tasks,  i.e.  those  tasks  which  must  be  or  are 
commonly  performed  by  more  than  one  soldier.  Collective  training  In  units  is 
categorized  according  to  the  element  performing  the  task,  e.g.  team,  squad, 
section,  platoon,  company,  etc.  Do  you  break  down  collective  training  In  units  In 
that  manner,  or  do  you  use  some  other  categorization? 


If  some  other  categorization  Is  used,  what  Is  It? 


What  collective  tasks  do  you  teach  at  the  lowest  level  of  organization  In  your  unit? 
How  do  you  conduct  that  training?  How  do  you  measure  performance? 


What  collective  tasks  do  you  teach  at  higher  levels  of  organization  In  your  unit? 
How  do  you  conduct  that  training?  How  do  you  measure  performance? 


What  percentage  of  your  training  time  do  you  spend  on  collective  tasks? 


Is  there  a  means  for  measuring  the  performance  of  the  soldiers  on  these  collective 
tasks?  If  not  all,  approximately  what  percentage  of  the  tasks  are  subject  to 
performance  measurement? 


How  Is  collective  performance  generally  measured? 


On  a  scale  of  one  to  five  (five  being  the  highest  rating)  how  would  you  evaluate  the 
performance  measurement  system  you  use  to  measure  collective  task 
performance  as  to; 

validity? _ 

reliability? _ 

accuracy/ _ 

fairness? _ 

effectiveness? _ 
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Do  you  use  a  pass/fall  measurement  system  or  a  graded  scale  system,  'f  ?  graded 
system,  is  it  norm-based  or  criterion-based. 

Are  training  devices  ever  used  in  measuring  collective  task  'Performance?  If  so,  is 
the  decision  on  whether  the  standard  has  been  met  made  automatically  by  the 
device? 


If  the  pass/fail  or  grading  decision  is  made  by  a  human  observer,  is  the  decision 
objective  or  suyective?  [Does  the  standard  Itself  require  judgment  on  the  part 
of  the  observer?! 


What  criteria  do  you  use  to  determine  if  the  soldiers  have  had  enough  training  on  a 
particular  collective  task? 


Who  established  the  criteria? 


What  happens  if  a  soldier  continually  fails  to  contribute  to  the  accomplishment  of 
the  platoon's  collective  tasks? 


Assuming  they  meet  the  appropriate  criteria,  how  do  you  determine  what  task  they 
get  trained  on  next? 


Do  you  ever  amend  the  conJi lions  and/or  standards  that  are  prescribed  by  whatever 
source  document  you  use? 


Are  there  collective  tasks  that  are  performed  in  your  unit  by  soldiers  with  MOS 

_ that  are  not  reflected  in  the  source  document  that  lays  out  the  collective 

tasks  those  soldiers  should  perform? 


If  yes,  who  makes  the  decision  to  Include  those  tasks?  How  do  you  fit  those  into 
your  training  schedule?  How  do  you  determine  the  appropriate  conditions  and 
standards? 


When  a  group  of  soldiers  completes  training  on  a  particular  task  and  has  their 
performance  measured,  how  are  those  results  used' 
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Are  there  any  collective  tasks,  critical  to  the  mission  accomplishment  of  your 
platoon,  that  the  soldier  is  not  trained  In  at  the  school  prior  to  his  Joining  the 
unit? 


If  yes,  what  are  those  tasks? 


Do  you  believe  those  tasks  should  be  trained  at  the  school? 


Do  you  know  why  those  tasks  are  not  trained  In  the  school? 


Are  there  any  collective  tasks  your  soldiers  are  trained  to  do  In  the  school  but 
cannot  perform  when  they  are  In  the  unit? 


Have  you  been  able  to  determine  why  they  are  unable  to  do  those  particular  tasks? 


nave  you  ever  talked  with  anyone  fron  the _ School  about  the  ability  of 

their  graduates  to  perform  their  collective  tasks  on  the  job? 


Do  you  think  the _ School  Is  effectively  training  the  M05 _ soldiers  in 

how  to  perform  collective  tasks? 


If  not,  what  should  they  do  differently? 


In  your  opinion,  should  the  schools  train  collective  tasks? 


In  your  opinion,  Is  the  link  between  the  units  and  the  schools  an  Important  one? 


Is  that  link  between  unit  and  school  accomplishing  anything? 


How  can  the  link  be  Improved? 


Of  all  the  decisions  that  are  made  which  affect  the  training  of  your _ 

who  makes  them,  what  percentage  of  the  decisions  that  are  made  are  made  bv 
them,  and  what  kinds  of  decisions  do  they  make?  [The  list  of  decision-makers 
may  Include  those  personnel  not  located  at  the  particular  Installation.) 


In  order  to  make  those  kinds  of  decisions,  what  kind  of  information  or  data  should 
they  have  on  which  to  base  those  decisions? 

Is  It  your  belief  that  they  are  getting  that  Information? 

How  can  the  data  from  the  performance  measurement  system  used  In  both  Individual 
and  collective  training  in  the  units  be  used  to  improve  the  training? 

Who  should  be  receiving  that  data? 

How  can  that  data  be  given  to  the  right  decision  makers? 
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APPENDIX  C 

UNIT  TRAINING  SURVEY  INSTRUMENT  II 


C-1 


The  Army  Reseerch  Institute  (ARi)  is  conducting  a  study  on  how  the  Army  conducts 
indivlduel  end  collective  treinlng  in  units  end  how  the  performence  of  both  Indlvlduel 
end  collective  tasks  Is  measured.  Your  assistance  in  completing  the  following 
questionnaire  will  help  in  that  study. 

Please  answer  the  following  questions  onty  as  they  pertain  to  ilOS  68B10  to  the 
best  of  your  ebiiity  by  circling  the  most  appropriate  answer.  If  an  answer  is  ‘other*, 
please  fill  in  the  blank.  Note  that  for  some  questions  more  than  one  answer  may  apply. 
Also  note  that  some  of  the  questions  ( 1  through  18)  deal  with  individual  tasks,  some 
(questions  19  through  37)  deal  with  collective  tasks,  and  the  final  five  questions  (38 
through  42)  deal  with  both  individual  and  collective  tasks.  When  you  have  finished, 
fold  this  paper  so  that  the  address  is  visible,  staple  it,  and  drop  in  the  mail  box  —  no 
stamp  is  necessary.  Thanks  for  your  cooperation. 

Your  duty  position? _ How  many  months  in  Job? _ 

1 .  Who  decides  what  individual  tasks  will  be  trained  on  in  your  unit? 

a.  SqdLdr  b.  PltSgt  c.  PltLdr  d.  CoCO  e.  BnCO  f.  Bde  g.  Div 

2.  What  document  would  you  go  to  in  order  to  see  the  list  of  individual  tasks  soldiers  with  MOS  68B 1 0 
are  required  to  perform? 

a.  Soldiers  Manual  b.  ARTEP  c.  MOS  Job  Book  d.  Unit  SOP  e.  Other _ f.  Don“t  Know 

3.  What  document  would  you  go  to  in  order  to  see  the  conditions  and  standards  associated  with  the 

individual  tasks? 

a.  Same  as  above  b.  Other _  c.  Don't  know 

A.  individual  Training  in  Units  is  often  referred  to  in  a  number  of  wa/s.  Listed  below  are  the 
categories  frequpf'tly  Mcsd  in  Army  training  publications.  Circle  the  ones  you  are  familiar  with  wx)  would 
use  when  talking  about  the  kind  of  individual  training  your  unit  does. 

a.  Integration  Tng  b.  Sustainment  Tng  c.  Enhancement  Tng  d.  Skill  Level  Progression  Tng 

e.  Transition  Tng  f.  Merger  Tng  g.  Reclassification  Tng  h.  Other _ 

5.  Do  you  know  what  individual  tasks  are  currently  being  trained  on  at  the  Aviation  Logistics  School  for 
soldiers  who  are  In  training  to  become  68B 1  Os? 

a.  Yes  b.  No 

6.  If  the  answer  was  yes,  how  do  you  know? 

a.  Saw  a  list  put  out  by  the  Aviation  Logistics  School 

b.  Other _ 

7.  What  percentage  of  your  training  time  do  you  spend  on  individual  tasks? 

a.O-2051  b.  20-40*  c.  40-60*  d.  60-80*  e  80-100* 

8.  What  percentage  of  the  individual  tasks  that  are  trained  are  subject  to  some  form  of  performance 
m?3Si:rement? 

a.  0-20*  b.  20-  40*  c.  40-60*  d.  60-  80*  e.  80-100* 
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9.  How  is  the  perfxmance  meosured  as  a  general  rule?  [more  than  one  answer  may  appfy] 


a.  SQT  b.  Fxmal  hands-on  performance  lest  c.  Formal  paper /pencil  test 
d.  Informal  observation  by  supervisor/leader  e.  Other _ 

1 0.  The  method  you  use  to  measure  performance  of  Individual  tasks  in  your  unit  accomplishes  what  it's 
supposed  to. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strngly  agree 

11.  The  method  you  use  to  measure  performance  of  Individual  tasks  in  your  unit  generally  provides 
consistent  results  over  a  period  of  time. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  a^ee 

12.  Most  of  the  soldiers  in  your  unit  wiuld  say  that  the  way  their  Individual  task  performance  is 
measured  is  fair. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  agree 


13.  From  your  point  of  view,  the  way  that  performance  of  individual  tasks  is  measured  in  your  unit  is 
effertive. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  agree 

1 4.  Which  of  the  following  best  describes  your  Individual  task  performance  measurement  system? 

a.  Go /no  go  [ 'Vou  passed"]  b.  Graded  scale  ["you  got  a  93"] 

1 5.  Do  you,  or  anyone  else  in  your  unit,  ever  change  the  Individual  task  conditions  and/or  standards 
from  what  is  written  in  the  DA  published  documents? 

a.  Always  b.  Sometimes  c.  Seldom  d.  Never 

1 6.  Are  your  soldiers  ( MOS  686 10)  required  to  perform  any  individual  tasks  that  are  not  included  in 
the  DA  published  dxuments? 

a.  Always  b.  Sometimes  c.  Seldom  d.  Never 

1 7.  Are  there  any  individual  tasks,  critical  to  the  accomplishment  of  your  unit's  mission,  that  you  know 
are  not  being  trained  at  the  Aviation  Logistics  school? 

a.  Many  b.  Some  c.  Very  few  d.  None 

18.  Are  there  any  Individual  tasks  that  686 1  Os  are  trained  on  at  the  Aviation  Logistics  School  but 

which  they  cannot  perform  once  they  are  in  the  unit? 

a.  Many  b.  Some  c,  Very  few  d.  None 
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1 9.  Who  decidBS  what  collective  tasks  will  be  trained  on  In  your  unit? 


a.  SqdLdr  b.  PltSgt  c.  PULdr  d.  CoCO  e.  BnCO  f.  Bde  g.  Div 

20.  What  (tocument  would  you  go  to  in  order  to  see  the  list  of  collective  tasks  soldiers  with  MOS  68B 1 0 
are  required  to  perform? 

a.ARTEP  b.  Unit  SOP  c.  Other _  1  Don't  Know 

2 1 .  What  dxument  would  you  go  to  In  order  to  see  the  conditions  and  standards  associated  with  the 

collective  tasks? 

a.  Same  as  above  b.  Other _  c.  Don't  Know 

22.  At  what  levels  of  organization  in  your  unit  have  you  participated  In  collective  training? 

a.  Crew  b.  Squad  c.  Section  d.  Platoon  e.  Company  f.  Battalion  g.  Brigade  h.  Division 

23.  Do  you  know  what  collective  tasks  are  currently  being  trained  on  at  tiie  Aviation  Logistics  School 
for  soldiers  who  are  in  training  to  become  68B 1  Os? 

a.  Yes  b  No 

24.  If  the  answer  was  yes,  how  do  you  know? 

a.  Saw  a  list  put  out  by  the  Aviation  Logistics  School 

b.  Other _ 

25  What  percentage  of  your  training  time  do  you  spend  on  collective  tasks? 

a.0-20*  b.  20-40X  c.  40-60^  d.  60-80*  e,  80-100* 

26.  What  percentage  of  the  Individual  tasks  that  are  trained  are  subject  to  some  form  of  performance 
measurement? 

a.0-20*  b.  20-  40*  c.  40-60*  d.  60-80*  e.  80-100* 

27  How  is  collective  task  performance  measured  as  a  general  rule?  [acre  than  one  answer  may 
apply] 

a.  External  ARTEP  evaluation  b.  Internal  ARTEP  evaluation  c.  Other _ 

28  The  method  you  use  to  measure  performance  of  collective  tasks  in  your  unit  accomplishes  what  it's 
supposed  to? 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  1  agree  e.  strongly  agree 


29.  The  method  you  use  to  measure  performance  of  collective  tasks  in  your  unit  generally  provides 
consistent  results  over  a  period  of  time. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  agree 
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30.  Most  of  the  soldiers  In  your  unit  would  say  that  the  wo/  their  coUective  task  performance  is 
measured  Is  fair. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  agree 

31.  From  your  point  of  view,  the  way  that  performance  of  collective  tasks  Is  measured  in  your  unit  Is 
effective. 

a.  strongly  disagree  b.  disagree  c.  somewhat  agree  d.  agree  e.  strongly  agree 

32.  Which  of  the  following  best  describes  your  collective  task  perfxmance  measurement  system? 

a.  00/ no  go  ["you  passed"]  b.  Graded  scale  ("you  got  a  93"] 

33.  Do  you.  or  anyone  else  In  your  unit,  ever  change  the  collective  task  conditions  and/or  standards 
from  what  Is  written  In  the  DA  published  documents? 

a.  Always  b.  Sometimes  c.  Seldom  d.  Never 

34.  Are  your  soldiers  (MOS  68B10)  required  to  perform  any  collective  tasks  that  are  not  Included  In 
the  DA  published  documents? 

a.  Always  b.  Sometimes  c.  Seldom  d.  Never 

35.  Are  there  any  collective  tasks,  critical  to  the  accomplishment  of  your  unit’s  mission,  that  you  know 
are  not  being  trained  at  the  Aviation  Logistics  School? 

a.  Many  b.  Some  c.  Very  few  d.  None 

36.  Are  there  any  collective  tasks  that  68B10s  are  trained  on  at  the  Aviation  Logistics  School  but 
which  the/  cannot  perform  once  they  are  In  the  unit? 

a.  Many  b.  Some  c.  Very  few  d.  None 

37.  In  your  opinion,  to  what  extent  should  collective  tasks  be  taught  at  the  Aviation  Logistics  School? 

a.  Heavy  emphasis  b.  Moderate  extent  c.  A  little  d.  Not  at  al  l 

38.  What  happens  to  the  results  of  performance  measurement  done  in  your  unit?  [more  then  one 
answer  nay  apply] 

a.  Used  to  decide  training  requirements  In  unit.  b.  Sent  to  the  Aviation  Logistics  School 
c.  Used  for  readiness  evaluations,  d.  Other _  e.  Don't  know 

39.  To  what  extent  are  training  devices  used  to  measure  performance? 

a.  Very  greet  extent  b,  Oreat  extent  c.  Moderate  extent  d.  Little  extent  e.  None 

40  When  training  devices  are  used  In  performance  measurement,  does  the  performance  get  recorded 
automatically  by  the  device,  or  Is  an  Instructor  required  to  record  the  results? 

a.  Automatically  by  device  b.  Instructor  must  record  c.  Some  of  each 
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41.  If  the  instructor  Is  required  to  record  the  results,  must  he  use  subjective  judgment  ( in  other  words 
two  different  Instructors  viewing  the  some  performonce  could  have  different  opinions  of  whether  the 
performance  was  a  ’go'  or  'no  go')  or  are  the  stwidords  objective  (not  subject  to  the  judgment  of  the 
instructor)? 

a.  Requires  subjective  judgment  b.  Objective  c.  Some  of  each 

42.  Have  you  ever  tellced  with  anyone  from  the  Aviation  Logistics  school  about  how  well  68B 1  Os  can  do 
their  Job? 

a.  Yes  b.  No  c.  No.  but  others  In  my  unit  have 

Thanks  for  your  help.  Please  fold  this  questionnaire  so  that  the  address  Is  visible, 
staple  It.  and  ih'op  In  the  mall  box.  No  stamp  Is  necessary. 
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APPENDIX  D 

DETAILED  DESCRIPTIONS  OF  SURVEY  RESULTS 


DETAILED  DESCRIPTIONS  OF  SURVEY  RESULTS 


This  appendix  gives  a  detailed  description  of  the  procedures 
used  to  determine  the  PMA  methods  for  the  six  MOSs.  It  also 
provides  the  detailed  findings  of  the  survey,  both  through 
narrative  text  and  then  through  the  use  of  the  PMA  matrices 
described  in  the  body  of  the  report.  The  appendix  first 
discusses  individual  skills  training  in  institutions,  followed  by 
collective  training  in  institution,  individual  skills  training  in 
units,  and  finally  collective  skills  training  in  units. 

Training  of  Individual  Skills  in  Institutions 

Tasks  Trained/Evaluated.  The  individual  tasks  selected  for 
training  in  the  institutions  are  generally  those  which  the 
individual  soldier  is  expected  to  accomplish  when  he/she  is 
assigned  to  a  unit,  although  in  some  instances  there  are 
discrepancies.  For  example,  some  tasks  may  be  trained  in  the 
institution,  as  evidenced  by  their  inclusion  in  the  POI  - 
although  the  soldier  is  apparently  not  expected  to  perform  those 
tasks  in  ■‘■be  unit,  since  these  tasks  are  not  included  in  the 
appropriate  soldier's  manual.  Conversely,  there  are  some  tasks 
that  the  soldier  apparently  is  expected  to  perform  in  the  unit 
but  which  are  not  trained  in  the  institution.  In  other  words, 
these  tasks  are  listed  in  the  soldier's  manual  but  NOT  the  POI. 
Thus  the  number  of  tasks  trained  is  not  necessarily  equa]  to  the 
number  of  tasks  the  soldier  will  be  required  to  perform  in  the 
unit.  Table  D-1  presents  the  number  of  MOS-related  tasks  trained 
in  the  schools  and  compares  it  to  the  number  of  MOS-related  tasks 
contained  in  the  appropriate  soldier's  manual  for  the  particular 
MOS.  Common  tasks  reflected  in  STP  21-1  SMCT,  the  common  tasks 
soldier's  manual,  are  not  included.  These  numbers  were  arrived 
at  by  inspection  of  the  respective  programs  of  instruction  (POIs) 
and  the  soldier's  manuals  for  Skill  Level  1. 

In  addition,  not  all  tasks  selected  for  training  in  the 
institution  are  trained  to  the  standards  that  will  be  required  in 
the  unit.  Each  course  POI  states  which  of  the  tasks  are  not 
trained  to  standard.  Table  D-2  shows,  for  each  of  the  six 
courses  looked  at  in  this  effort,  the  number  of  tasks  not  trained 
to  standard  and  compares  that  number  with  the  total  number  of 
tasks  reflected  in  the  POI. 

The  school  survey  respondents  indicate  that  this  is  generally 
due  to  a  lack  of  available  training  time  in  the  institutions.  By 
way  of  elaborating  on  this  point,  the  respondents  cite  the 
pressures  on  the  school  to  get  soldiers  out  of  the  training 
"pipeline"  and  into  the  units  as  rapidly  as  possible.  Both  the 
school  and  unit  survey  respondents  claim  that,  to  a  certain 
extent,  these  discrepancies  are  remedied  by  means  of  the 
integration  training  the  soldier  receives  when  he  arrives  at  the 
unit.  However,  since  there  is  no  formal  documentation  of  the 
specific  tasks  trained  during  this  integration  training,  no 
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Table  D-1 


Number  of  Tasks,  by  MOS,  in  Programs  of  Instruction 
compared  with  Soldier's  Manuals 


POI 

SM 

19EtO 

64 

175 

19K10 

64 

813^ 

63E10 

135 

b 

419 

63N10 

N/A^ 

480^ 

67Y10 

52 

66 

68B10 

86^ 

105 

®  While  the  Soldier's  Manual  lists  only  83  MOS-specIfic  tasks,  the  Job 
B«k  lists  96  MOS-specIflc  tasks. 

°  Includes  64  "duty  position  tasks"  and  355  "related  technical  tasks." 

^  Number  of  tasks  ore  not  Identified  In  POI  but  are  believed  to  be 
approximately  135,duetoslmllarlty  with  M0S63E10. 

°  Includes  50  "duty  position  tasks"  and  430  "related  technical  tasks." 

®  In  addition  to  the  86  tasks,  POI  reflects  40  "other  tasks  and 
subjects"  which  are  grouped  under  the  rubric  "enabling  skills." 
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Table  D-2 


Number  of  Tasks,  by  MOS.  Not  Trained  to  Standard  In 
Schools,  compared  with  Total  Number  of  POI  Tasks 


Not  Trained 
to  Standard 

Number  of 
POI  Tasks 

19E10 

13 

64 

19K10 

10 

64 

63E10 

0 

135 

63N10 

N/A 

N/A 

67YI0 

•  11 

52 

68B10 

1 

86 
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definitive  conclusion  could  be  made  as  to  whether  all  of  those 
tasks  are  presented  during  the  integration  training  phase. 

As  the  data  in  Table  D-2  show,  the  number  of  tasks  trained  in 
the  school  is  less  than  the  number  of  tasks  the  soldier  is 
expected  to  be  able  to  perform  in  the  unit.  The  survey 
respondents  indicated  this  is  the  case  because  the  schools,  faced 
with  constraints  on  available  time  to  train,  can  train  only  the 
most  critical  tasks  the  soldier  will  be  required  to 
perform  when  assigned  to  a  unit.  The  shortfall  is  significantly 
different  depending  on  the  particular  MOS.  In  the  case  of  the 
two  Career  Field  63  MOSs,  the  soldier's  manuals  cite  a  large 
number  of  "related  technical  tasks"  which  according  to  the 
manuals  can  be  accomplished  if  the  soldier  can  perform  the  "duty 
position"  tasks  to  the  required  standards. 

A  specific  example  may  explain  the  seemingly  large  disparity 
between  the  number  of  tasks  trained  in  the  school  and  the  number 
of  tasks  the  soldier  will  be  called  on  to  perform  in  the  unit;  it 
also  helps  explain  the  relationship  of  these  "duty  position 
tasks"  and  "related  technical  tasks."  Task  #  091-499-1052 . 

Replace  generator/alternator  fM35-series  vehicle)  is  a  duty 
position  task  cited  in  the  soldier's  manual  for  63E10;  several 
"related  technical  tasks",  which  appear  to  be  very  similar, 
include  #  091-499-1113  Replace  generator  (60  Amp.  25  Amp) .  ^MlSl 
series  vehicle) ;  and  #  091-499-1153.  Replace  regulator  (M35 
series  vehicle) .  While  it  might  be  argued  that  these  are  not 
discrete  tasks  but  rather  the  same  task  performed  ^mder  different 
conditions,  i.e.,  on  different  equipment,  the  Ordnance  School  - 
which  is  the  proponent  for  the  MOS  -  has  obviously  chosen  to 
differentiate  among  the  tasks  and  has  given  each  a  unique  number. 
Other  schools  apparently  do  not  necessarily  identify  tasks  to  the 
same  level  of  detail;  this  would  explain  to  some  extent  why  the 
number  of  soldier's  manual  tasks  for  63E10  and  63N10  is  so  large 
in  relation  to  the  number  of  tasks  reflected  in  the  POI  and 
soldier's  manuals  for  the  other  MOSs. 

Inspection  of  the  POIs  and  the  interviews  with  school 
training  development  personnel  reveal  that  performance  is 
routinely  measured  during  the  courses  at  the  schools,  although 
there  are  slight  discrepancies  between  the  number  of  tasks 
trained  and  the  number  of  tasks  evaluated.  Table  D-3  presents  the 
number  of  tasks  that  are  evaluated  for  each  of  the  MOS  producing 
courses  surveyed  and  compares  the  number  of  tasks  evaluated  to 
the  number  of  tasks  trained.  In  the  two  instances  of  one-station 
unit  training  (OSUT)  -  19E10  and  19K10  -  the  number  of  tasks 
related  to  the  basic  training  portion  of  the  courses,  i.e.,  the 
common  soldier  tasks,  are  not  included;  only  the  MOS  specific 
tasks  are  considered  in  order  to  make  those  two  courses  more 
comparable  to  the  other  courses. 

As  can  be  seen  from  Table  D-3,  data  are  not  available  to 
support  any  generalizations.  Only  in  the  case  of  the  two  Armor 
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T&ble  D-3 


Number  of  Tasks  Evaluated  in  School,  by  M05, 


Compared  with  Number  of  Tasks  Trained 


Tasks 

Evaluated 

Tasks 

Trained 

19E10 

61 

64 

19K10 

62 

64 

63E10 

N/A 

135 

63N10 

N/A 

N/A 

67Y10 

N/A 

68BI0 

N/A 

36 
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Crewman  MOSs  where  the  course  test  and  evaluation  plans  were 
available  could  specific  findings  be  made.  As  could  be  expected, 
the  number  of  tasks  evaluated  is  fewer  than  the  number  cf  tasks 
trained.  In  the  case  of  the  other  MOSs,  although  hard  data  were 
not  available,  there  were  indications,  based  on  the  results  of 
the  interviews  with  school  personnel,  that  tasks  are  evaluated  on 
a  sampling  basis,  i.e.,  not  every  task  trained  is  evaluated; 
those  that  are  evaluated  vary  from  course  iteration  to  iteration. 
In  some  cases  the  POIs  reinforced  this  perception  on  the  part  of 
the  survey  respondents.  For  example,  the  POI  for  the  63E10 
course  states  that  many  examinations  require  the  student  to 
perform  only  "selected  critical  tasks"  from  the  lessons  covered 
by  the  exams. 

The  authors  selected  one  particular  course — the  19K10 
course — in  order  to  take  a  closer  look  at  one  MOS  and  determine 
the  relationships  between  tasks  trained  and  those  evaluated. 

This  course  was  selected  solely  on  the  basis  of  the  amount  of 
information  available  -  the  authors  were  able  to  collect  more 
documentation  on  this  course  than  on  any  of  the  others.  In 
addition  to  having  access  to  the  Program  of  Instruction,  the 
authors  were  able  to  review  the  Soldier's  Manual,  the  Job  Book, 
and  the  Test  and  Evaluation  Plan  for  the  MOS  19K10  course.  This 
afforded  the  opportunity  to  cross-check  the  tasks  trained  in  the 
course,  the  tasks  evaluated  during  the  course,  and  the  tasks 
required  on  the  job. 

The  detailed  examination  of  the  19K10  course  reveals  that  the 
POI  lists  64  discrete  MOS-related  tasks  trained  of  which  eight 
are  not  trained  to  standard.  Of  those  64  tasks  trained,  only  two 
are  not  evaluated.  These  two  tasks  are  shown  in  Table  D-4 . 

Since  the  Armor  School  evaluates  the  physical  fitness  of  its 
students  -  an  Army  req'uirement  -  it  appears  that  the  listing  of 
Task  171-123-1005  as  not  being  evaluated  is  merely  an 
administrative  oversight,  and  the  number  of  tasks  not  evaluated 
is  more  likely  only  one. 

Table  D-5  lists  five  tasks  that  are  listed  in  the  Test  and 
Evaluation  Plan  for  the  course  but  which  are  not  reflected  in  the 
Program  of  Instruction.  The  inclusion  of  the  task  "Establish 
Tank  Firing  Positions"  in  this  as  well  as  in  Table  D-4  is  due  to 
the  misnumbering  of  the  task  in  one  of  the  two  documents.  Thus 
it  appears  likely  that  this  task  is  trained  AND  evaluated.  The 
listing  of  the  remaining  four  tasks  in  Table  D-5  is  more  than 
likely  due  to  the  differing  publication  dates  of  the  two 
documents.  It  seems  unlikely  that  the  evaluation  of  a  task  for 
which  the  soldier  had  received  no  training  would  go  unnoticed, 
and  unresolved,  past  one  iteration  of  the  course. 

Another  minor  discrepancy  noted  was  the  fact  that  of  all  the 
tasks  listed  in  the  POI  for  the  MOS  19K10  course  (there  are  159 
tasks,  all  but  64  of  which  are  common  tasks,  i.e.,  not 
specifically  related  to  the  MOS) ,  42  do  not  appear  in  any  of  the 
training  publications  produced  for  the  units.  In  other  words, 
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Table  D-4 

19K10  Tasks  in  POI  Not  Evaluated 

Task  Number  Task 

171-1 23- 1 005  Attain  an  Aporopriate  Level  of  Physical  Fitness 
171-1 23-  ’  008  Establish  Tank  Firing  Positions 


Table  D-5 


Tasks  Evaluated  in  19KI0  Course  but  Not 
Reflected  in  POI 

Task  Number  Task 

031-123-1008  Establish  Tank  Firing  Positions 

07 1  -325-0001  Employ  a  Live  Fragmentation  Grenade 


08 1  -83 1-1012  Give  First  Aid  to  a  Blister  Agent  Casualty 

08 1  -93 1  - 1 033  Apply  a  Dressing  to  an  Open  Head  wound 


171-126-1065  Inspect  Hydrauiir<=;  on  ar,  M 1 /M 1  a:  Tank 
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those  42  tasks  do  not  appear  in  either  the  soldier's  manual  for 
19K10,  the  job  book,  or  the  soldier's  manual  of  common  tasks. 

One  •^ould  infer  from  this  that  tasks  are  trained  in  the  19K10 
course  that  are  not  required  to  be  performed  by  the  soldier  in 
his  ^ob.  A  closer  examination  of  these  42  tasks  (Table  D-6)  does 
not  necessarily  support  such  an  inference.  Thirty-seven  of  the 
tasks  are  those  generally  associated  with  Basic  Combat  Training 
(e.g.,  identify  rank,  execute  drill  movements,  react  to  the 
sounding  of  "Retreat,"  etc.)  and,  while  not  specifically  cited  in 
training  support  material,  are  commonly  required  as  tasks  which 
are  appropriate  for  all  soldiers.  The  remaining  five  tasks 
(171-126-1054,  -1055,  -1057,  -1058,  and  -1060)  are  specific  to 
the  MOS.  There  is  no  explanation  in  any  of  the  documents 
reviewed  as  to  why  these  42  tasks  are  not  reflected  in  the 
training  support  material  mentioned.  Inspection  of  the  tasks 
suggest  that  their  inclusion  in  the  POI  is  appropriate.  The  five 
MOS-related  tasks  appear  to  be  appropriate  for  inclusion  in  the 
training  support  material  as  well. 

Finally,  there  are  a  number  of  tasks  that  are  required  of  the 
19K10  soldier  in  the  unit  but  are  not  trained  in  the  school. 

These  tasks  are  listed  in  Table  D-7.  As  previously  indicated, 
the  assumption  has  to  be  made  that  these  tasks  are  addressed  in 
integration  training  once  the  soldier  is  assigned  to  a  unit,  but 
there  is  no  docume.ntation  to  check  the  validity  of  this 
assumption. 

Returning  to  the  investigation  of  all  the  selected  MOSs,  it 
appears  that  the  procedures  used  to  decide  which  tasks  are  not 
trained  in  the  school,  thus  leaving  the  training  of  those  tasks 
to  the  unit  during  integration  training,  may  be  somewhat  flawed. 
TRADOC  Reg  350-7,  A  Systems  Approach  to  Training,  requires  the 
schools  to  conduct  front-end  analyses  of  individual  job 
requirements,  publish  soldier's  manuals  outlining  the  tasks  for 
which  the  soldier  is  responsible,  and  select  tasks  for  training 
in  the  school  (reflected  in  the  POI  for  the  particular 
MOS-producing  course) .  Thus  the  task  selection  process  is 
predicated  on  the  analysis  of  the  unit's  mission  and  the 
individual  skills  and  tasks  that  soldiers  with  that  particular 
MOS  need  to  possess  or  be  able  to  perform  in  order  for  the  unit 
to  accomplish  its  mission.  However,  there  are  some  indications 
that  the  policies  are  not  being  fully  implemented.  An  article  by 
senior  staff  officers  at  HQ  TRADOC  (Duncan  &  Hartjen,  1985)  and 
discussions  with  senior  officials  of  the  Army  Training  Board 
(ATB)  indicate  that  front-end  analyses  are  not  currently  being 
accomplished  by  the  TRADOC  schools  in  strict  accordance  with  the 
established  policies.  Thus  it  is  possible  that  the  task 
selection  methodology,  and  the  measurement  of  individual 
performance  in  the  schools,  is  less  than  what  is  called  for  in 
TRADOC  Reg  350-7  with  the  result  that  there  may  be  a  disconnect 
between  tasks  trained  and  measured  in  the  schools  and  those  that 
are  required  to  be  performed  in  the  job  environment. 
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Table  D-6 


Tasks  Listed  in  1 9K 1 0  POI  but  not  in  FM  21-1  (Common  Skills), 
Soldier's  Manual  or  Job  Book 


Task  Number 


Task 


071-503- ’008 
071-326-0001 
071-326-O0C2 
071-326-0010 
071-326-0011 


Decontaminate  inaiviocial  EaciiDment 
Set  Up  and  Strike  a  She!te''-Half  Tent 
Identify,  Prepare,  and  Wear  Army  issue  Uniforms 
Report  to  an  Offirf-  indoor? 

Report  10  an  Ufficer  Outdoors 


071-326-0012 

071-326-0013 

071-326-0014 

071-326-0015 

071-326-0016 

React  to  an  Approaching  Officer 

Rear*  to  an  Approaching  NCO 

Identify  Rank 

React  to  Passing  Colors 

Reoct  10  the  Playing  of  '"’‘he  National  Anthem  and  "To  the  Colors 

071-326-0017 

071-326-0018 

071-326-0019 

071-326-0020 

071-326-0030 

React  to  tne  Playing  of  "The  Army  Song 

Peact  to  the  Souno.ng  of  "Reveille" 

React  to  the  Sounding  of  "Retreat" 

React  to  an  Oh^'icer  Entering  a  Bu'Id’ng 

E>iecute  Drill  Movements  Without  Arms 

071-326-0031 

071-326-0032 

071-326-0033 

071-326-0034 

071-327-0002 

Execute  Drill  Movements  With  Arms 

Perform  as  a  Squad  Member  During  the  Conduct  of  Souad  Dril' 

Per  form  as  a  Squad  Member  During  the  Conduct  of  Platoon  Dr:l; 

Her-'orm  as  3  Squad  Member  During  the  Conduct  of  Company  Drill 

Compete  a  iS-’S  Mile  (20-24  sm)  Tactical  Fool  March 

07':-32‘'-00C3 

07 1-32/ -0004 
071-328-001 1 
o7i-:'2e-ooi2 
071-328-0013 

Conditioning  Obstacle  Course 

Confidence  Obstacle  Course 

Prepare  for  Working  Inspection 

R-epare  ^or  ai.  in-Ouarters  inspection  (Standby  m  Dutv  Uniform) 
R'ecar  e  for  :r.  Ranks  Inroection 

071-328-0014 

07i-33i-y050 

0~  1-331 -005! 

071  -331-0052 
071-331-0053 

Prepare  for  in-Ouarters  inspection  (Fu'l  Fieid  Lay-Out) 

React  to  an  'r.opeclmg  (Officer 

Summon  Comn-anoer  of  Reue^ 

Challenge  Unki  own  Personss) 

Apprehend  an  Intruder 

071-331-0054 
061  -631-1035 
061-831-1036 
06 ’-85 1-1037 
061-031-1036 


Stand  an  Informal  O-iaro  Hount 
P^'otect  Yourself  Against  Meat 
Protect  Yourself  Against  Cold 
Protect  Yourse'f  Agamst  Bding  insects 
Protert  Yourself  From  Diarrnea  arid  Dysenlei'y 


06  1-831-1 039 
171-126-1054 
171-126-1055 
17  1-126- ’057 


Pracl'ce  Personal  Hygmne  to  Maintain  Fitness 

Adjust  Operating  Cam  on  the  ’OSmm  Ham  Gun  on  an  P  ’  Tank 

Pef.  an  n  1  /Ml  1 A  1  '"ank 

Unlock  Stuck  lOnrit mg  P^akes  on  an  n  1 /M  1  A  i  Tank 
Servic^  the  Air  indjctior  System  or,  an  n;:, /MlAl  Tank 


171- 125- 1060 
171  126-1090 


'Prepare  an  hll/rdlAl  Tank  for  Rower  Pack  Removal 
Perform  Operator  Maintenance  and  Layout  of  Bn 
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Table  D-7 

1  9K  1 0  Soldier's  Manual  or  Job  Book  Tasks  ( Excluding  Those 

Listed  in  FM  21  -  1 )  Not  Trained  in  I9K10  Course 

Task  Number 

Task 

05  :  -505-3006 

Conduct  Partial  Decontamination 

05 '-192- ’00' 

Install/Remove  the  Ml 4  Blasting  Antipersonnel  Mine 

05:-i92-i002 

instail/Pemove  the  M16A1  Bounding  Fragmentation  Antipersonnel  Mine 

05  '  -  1 92- : 0 1 4 

install/Pemove  US  Antihandling  Devices  on  Antitank  Mines 

05 '-192- '023 

Locate  Mines  Using  the  AN/PSS-1  1  Mine  Detector 

051-192-1024 

Locate  Mines  Using  the  AN/PRS-7  Mine  Detector 

06 ’ -263-6003 

Ca'i  for/Adjust  Indirect  Fire 

071-315-0030 

Place  an  AN/PVS-5  (Night  Vision  Goggles)  Into  Operation 

07: -3 15-003; 

Perform  Operator  Maintenance  on  the  AN/PPVS-5  (Night  Vision  Goggles' 

.'7  1  -73  1  -0502 

Process  fsoown  or  Suspected  Enemy  Personnel/Documents/Equipment 

^  i 3-557- '050 

Mount  Padio  Set  AN/VPC-46 

• '3-557-3077 

Perform  Operator's  PMCS  on  Kaoio  bets  AN/VPC-12  or  -47,  etc 

I  1 3-565-4023 

Pepa'r  Field  Wire 

■  '  'j'j6 

instaM  Padio  Set  Control  Group  AN/GRA-SDC) 

'  3-522-2004 

Operate  Pad'o  Set  Control  Group  AN/GRA-39(*) 

i7'-!::3-:090 

PemorT'  Preventive  Maintenance  on  Basic  Issue  Items  (BID 

’7  ■-•26- 'O' : 

TrouP'esnoot  the  AN/VVS-2  Night  Vision  Viewer  on  an  M 1  T ank 

'7 '-’26- '013 

TrouPlesnoot  the  Engine  on  an  Ml  Tank 

26-  '  0 '4 

T^ouPleshoot  the  Transmission  on  an  Ml  Tank 

1  '  '  ^0  ■  '^2  1  J 

Trout'esnoot  tne  Personnel  Heater  on  an  Ml  Tank 

I -■-126-'';  16 

TrouCiesnoot  the  Gas  Particulate  Filter  Unit  on  an  Ml  Tank 

’7'-'26-'031 

Troubleshoot  the  Fire  Control  System  on  an  Ml  Tank 

’7- -'26- '032 

Troupieshoot  the  105mm  Mam  Gun  on  an  Ml  Tank 

1  7'  -  •26-103:’ 

Perfor-'-,  Ope'^ator  Maintenance  on  Ml  Tank  Periscopes 

171-126-1040 

Evacuate  a  Wounded  Crewmar  from  an  Ml  Tank 

•71- '26- '040 

■  ep'ace  a  Thrown  Track  on  an  Ml  Tank 

■7:-'26-'05C 

Zc.'‘-xrr.  ^ar.ua'  Extraction  of  a  Mam  Gun  Round  on  an  M  1  Tarik 

'71- 126-' 051 

■'.'ce-at^  ’ne  AN/VVS-2  Night  Vision  Viewer  m  Loader's  Hatch  on  an  Ml  Tank 

•5'-  ^i;6-  :5C  ■ 

App  y  the  Customs  and  Laws  of  War  Governing  Forbidden  Targets,  Tactics,  et; 

'r  '-O'-t-lOOO 

Apply  the  Customs  and  Laws  of  War  Governing  Treatment  of  Captives,  etc 

16  •-■906- 1503 

Apply  the  Customs  and  Laws  of  War  Governing  Protection  of  Civilians,  etc 

16' -906- 1504 

Apply  the  Cusonns  and  Laws  of  War  Governing  Prevention/Reportmg  of 

Criminal  Acts 
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The  TRADOC  policy  of  requiring  that  the  tasks  selected  for 
training  and  evaluation  in  the  institution  be  based  on  an 
analysis  of  the  tasks  required  of  the  soldier  in  the  job 
environment  appears  to  be  sound  if  not  without  flaw  in  its 
implementation.  Several  research  efforts  provide  evidence  that 
methodologies  for  developing  training  PMA  measurement  systems 
based  on  job-requirements  are  effective.  For  example.  Burroughs 
(1985)  provides  criterion  performance  measures  for  reliable  tests 
of  non-procedural  Ml  tank  driver  skills  that  could  serve  as 
standards  for  tank  driver  simulator  training.  Harper  and  Gutman 
(1981)  cite  specific  maintenance-related  performance  measures  for 
use  in  developing  and  implementing  an  Army  Maintenance 
Performance  System.  Biers  and  Sauer  (1983)  focused  on  Ml  Abrams 
gunnery  performance  and  used  an  aptitude  measurement  methodology 
to  design  job  sample  tests  (both  computer-based  and  hands-on)  for 
armor  crewmen.  The  common  feature  of  these  methodologies  is  that 
the  tasks  that  are  trained  are  based  on  the  tasks  that  are 
performed  in  the  job  environment. 

Despite  the  discrepancies  noted  in  the  tasks 
trained/ evaluated  in  the  institution  and  tnose  required  in  the 
unit,  it  would  appear  that  for  rhe  most  part  the  tasks  reflected 
in  the  POIs  are  generally  in  consonance  with  those  that  are 
performed  by  the  soldiers  on  the  job.  It  is  further  assumed  that 
similar  findings  would  be  made  if  all  MOS-producing  courses 
throughout  the  TRADOC  school  system  were  surveyed. 

Performance  Standards.  The  standards  used  for  measuring 
individual  task  performance  in  the  institutions  surveyed  are 
almost  without  fail  process-oriented  rather  than 
product-oriented.  Process  in  this  instance  refers  to  'how'  a 
soldier  performs  a  particular  task;  this  is  in  contrast  to  a 
product-oriented  approach  which  grades  the  outcome  of  the  task, 
or  'what'  the  soldier  has  done.  An  example  can  be  found  in  the 
task  of  boresighting  a  machine  gun  on  an  M60  tank.  The  standard 
contained  in  the  soldier's  manual  for  this  task  states  that  the 
soldier  has  satisfactorily  performed  the  task  if  the  center  of 
the  barrel  and  the  center  of  the  infinity  sight  reticle  are  on 
the  same  aiming  point.  This  is  a  product-oriented  approach.  The 
soldier's  manual  also  provides  an  evaluation  guide  that 
identifies  31  steps  which  the  soldier  must  pass  in  order  to  get  a 
GO  on  the  task.  This  is  a  process-oriented  approach.  The  fact 
that  tiie  institutions  tend  to  use  the  process-oriented  approach 
might  be  expected,  since  the  schools  are  training  MOS  skills  that 
have  not  previously  been  learned. 

Generally,  the  POIs  refer  to  the  standards  contained  in  the 
applicable  soldier's  manual  or,  especially  in  the  maintenance 
related  cou  ^es,  to  a  particular  technical  manual.  Inspection  of 
the  soldier's  manuals  reveals  that  these  publications  contain,  in 
addition  to  a  standards  statement',  a  list  of  performance 
measures  under  the  rubric  "E  aluation  Guide."  In  almost  every 
instance,  the  trainer/ evaluator  is  instructed  to  use  the 
performance  measures  to  evaluate  the  proficiency  of  the  soldier 
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in  accomplishing  a  particular  task.  Following  the  list  is  a 
statement  which  provides  guidance  to  the  evaluator.  As  an 
example,  STP  17-19E1-SM  contains  the  following  statement  for 
every  task:  "Score  the  soldier  00  if  all  steps  are  passed  (P) . 
Score  the  soldier  NO-GO  if  any  step  is  failed  (F) .  If  the 
soldier  fails  any  step,  show  what  was  done  wrong  and  how  to  do  it 
correctly."  (p.  2-5,  2-13,  2-16,  2-19,  etc.)  This  statement  can 
be  found  for  almost  every  task  in  every  soldier's  manual 
investigated,  thus  providing  evidence  that  process-oriented 
perfoirmance  standards  are  the  norm,  rather  than  the  exception, 
when  measuring  individual  performance  in  the  schools. 

Thus,  as  a  general  rule,  instead  of  evaluating  the  soldier  on 
whether  he  can  accomplish  a  task  to  the  specified  criterion  or 
standard,  he/she  is  evaluated  on  whether  each  and  every  step  of 
the  task,  as  described  in  the  evaluation  guide,  is  performed 
correctly.  Often,  the  evaluation  guide  requires  that  the 
performance  of  the  procedural  steps  be  in  the  proper  order,  which 
is  further  evidence  of  a  process-orientation. 

It  is  the  opinion  of  the  authors  that  the  process  approach  is 
probably  a  proper  orientation,  given  the  fact  that  the  quality  of 
the  institutional  training  program  can  best  be  evaluated  through 
the  evaluation  of  the  behavior  of  the  soldiers  undergoing  the 
training.  As  Popham  pointed  out  in  1971,  evaluation  and 
recording  of  the  learner's  behavior,  as  opposed  to  the  product  of 
the  behavior,  is  necessary  in  order  to  subsequently  evaluate  the 
quality  of  the  training  program  itself.  Popham  went  on  to  state 
that  "all  learner  behavior  requires  recording  in  order  to  be  used 
as  a  criterion  me  \sure  for  educational  evaluation."  Thus,  in  an 
institutional  training  environment,  especially  when  presenting 
entry  level  training,  process  orientation  is  more  important  and 
valuable  than  product  orientation. 

Performance  Measurement  Methodologies.  Training  PMA  during 
individual  training  in  the  institution  is  a  combination  of 
formal,  i.e.,  scheduled,  evaluations  -  conducted  at  the 
conclusion  of  a  block  of  instruction,  following  a  group  of  blocks 
of  instruction  (known  as  'gate'  tests)  and  as  end-of-course 
comprehensive  tests  -  and  informal  evaluations  by  the  instructor 
during  practical  exercises  conducted  during  the  presentation  of 
instruction.  With  respect  to  the  formal  evaluations,  the 
methodology  is  primarily  hands-on,  as  oppos'^  ’  to  written  (i.e., 
paper  and  pencil  tests)  .  This  appears  conscious  matter 

of  policy  at  the  TRADOC  schools.  For  exam^^e,  the  Armor  School 
requires  justification  for  any  evaluation  reflected  in  the  POI 
that  is  NOT  hands-on;  the  Directorate  of  Evaluation  and 
Standardization  challenges  any  proposed  written  evaluation  as  a 
matter  of  course.  As  evidence  of  this  policy,  one  that  is 
assumed  to  be  common  in  at  least  some  other  TRADOC  schools.  Table 
8  presents,  by  surveyed  MOS,  the  number  of  hours  devoted  in  the 
course  to  formal,  scheduled  written  evaluations  and  hands-on 
evaluations . 
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Table  D-8 


Hours  Devoted  to  Written  Evaluations,  by  M05, 
Compai  ed  with  Hands-on  Evaluations 


Written 

Evaluation 

Hands-on 

Evaluation 

19EI0 

0^ 

30  ^ 

19K10 

0 

ZO^ 

63E10 

5  6 

62  7 

63N10 

6.2 

63  7 

67Y10 

O 

•49 

68B10 

s 

67 

^  Excludes  2  hou''  wntlen  exam  on  General  Military  Sijbjects. 

^  Exc‘ude:  ^  hours  Priysicai  Readiness  Test  and  16  hours  devniph 
to  Gate  !  and  Gate  1)  examinations  wnch  are  not  MOS-soeciMc, 

^  Excludes  2  hour  exam  on  General  Military  Subiects,  ^  nours 
Physical  Readiness  Test  and  1 6  nours  devotee  to  Gates  :  ana  n 
examinations  A-nicn  are  not  MOS-SDecil'ic 
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with  respect  to  the  written  evaluations,  the  GO/NO  GO 
criterion  is  generally  based  on  the  soldier's  passing  a  total  of 
70%  of  the  items  presented  in  the  test.  In  the  case  of  hands-on 
tests,  however,  the  soldier  receives  a  GO  for  the  task  only  if 
all  the  steps  listed  in  the  Evaluation  Guide  are  passed.  The 
informal  evaluations  made  by  the  instructors  during  practical 
exercises  likewise  is  process-oriented,  wherein  the  instructor 
judges  the  ability  of  the  soldier  to  accomplish  the  various 
procedural  steps  of  the  task  in  the  proper  sequence.  This  policy 
is  specified  not  only  in  the  POIs  reviewed  during  the  study 
effort,  but  is  reflected  in  the  interviews  with  the  individuals 
surveyed  at  the  schools.  Again,  as  discussed  in  the  previous 
section,  this  is  in  keeping  with  the  general  notion  that  the 
evaluation  process  is  important  in  evaluating  the  training  system 
or  course  of  instruction  as  well  as  providing  the  individual 
soldiers  and  their  instructors  a  means  of  identifying  areas  where 
remedial  training  is  required. 

Use  of  Performance  Measurement  Data.  As  TRADOC  Reg  350-7,  A 
Systems  Approach  to  Training,  points  out,  "evaluation  is 
pointless  unless  action  is  taken  to  correct  deficiencies  and 
discrepancies  revealed  by  that  process,  and  the  management 
function  of  providing  for  quality  control  is  equally  important 
during  each  phase  of  the  training  process."  (p.  14).  In 
implementing  this  philosophy  of  evaluation,  the  TRADOC  schools 
surveyed,  and  by  inference  all  TRADOC  schools,  evaluate  the 
performance  of  the  soldiers  in  their  courses  and  use  the  results 
of  that  evaluation  process  for  a  variety  of  purposes. 

In  any  discussion  of  the  uses  to  which  the  training  PMA  data 
are  used,  it  is  important  to  include  the  audit  trail  of  those 
data.  There  is  little  evidence  either  from  previous  research 
results  or  the  results  of  the  survey  to  suggest  that  data  other 
than  the  final  grades  are  captured.  For  example,  if  a  soldier 
receives  a  GO  the  first  time  he/she  is  evaluated,  while  a  second 
soldier  receives  a  GO  only  after  four  attempts,  the  data  captured 
do  not  dif ;'erentiate  between  the  two;  the  second  soldier  receives 
remedial  training  until  a  GO  is  attained.  While  such 
differentiation  is  immaterial  to  a  decision  on  the  ability  of  the 
soldier  to  perform  a  task,  it  is  very  material  to  the  evaluation 
of  the  program  of  instruction.  Were  these  data  to  be  captured, 
much  could  be  learned  about  the  training  effectiveness  and 
efficiency  of  the  training  system,  an  area  which,  while  not 
totally  ignored  in  the  schools,  could  apparently  be  improved. 

The  survey  respondents  indicate  that  training  PMA  results  are 
used  to  determine  what,  if  any  remedial  training  is  required  for 
the  soldier  to  get  a  GO  on  each  task  evaluated.  In  those 
instances  where  the  soldier  gets  a  NO  GO,  he  or  she  is  given 
remedial  training,  often  one-on-one  with  the  instructor, 
and  given  the  opportunity (ies)  to  perform  the  task  successfully. 
Should  a  soldier  continue  to  fail,  attempts  are  made  to  isolate 
the  particular  problem.  As  a  last  resort,  the  soldier  is 
eliminated  from  the  course  and  reclassified  or  else  is  eliminated 
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from  the  Army,  although  this  apparently  is  very  rare.  This 
remediation  and  successful  performance  on  each  task  is  crucial  in 
the  institutional  environment  since  the  soldier  must  be 
proficient  in  all,  or  almost  all,  tasks  trained  in  order  to 
graduate  the  course  and  be  awarded  the  MOS.  Previous  research 
(Maxey,  1985;  Sticha,  Edwards,  et  al.,  1984;  and  Fineberg, 

Meister  &  Farrell,  1978)  indicates  that  remedial  training  based 
on  the  results  of  task  training  PMA  is  both  appropriate  and 
effective;  the  survey  revealed  that  this  use  of  training  PMA  data 
is  acknowledged  by  all  the  respondents. 

Other  uses  are  made  of  the  results  of  the  training  PMA  data. 
TRADOC  Reg  350-7  states  that  "feedback  is  used  to  evaluate  the 
program,  assess  the  quality  of  soldiers'  performances  and  check 
the  organization's  responsiveness  to  training  needs."  (p.  14). 

In  implementing  this  policy  at  the  Armor  School,  the  department 
of  Evaluation  and  Standardization  has  an  automated  data  base  that 
contains  the  results  of  the  training  PMA.  In  light  of  the  fact 
that  the  system  calls  for  remedial  training  for  those  soldiers 
who  fail  to  meet  the  standard,  and  that  ultimately  almost  all 
soldiers  pass,  it  is  questionable  as  to  how  useful  the  data  base 
is.  The  Armor  School  personnel  interviewed  state  that  data 
concerning  first  and  final  go/no-go  attempts  are  being  captured. 
If  a  soldier  fails  on  an  initial  attempt,  data  are  gathered  on 
subsequent  and  final  attempts.  Furthermore,  if  there  is  greater 
than  20%  failure  rate  a  close  investigation  of  the  training 
itself  is  undertaken,  and  the  necessary  adjustments  made. 

The  use  of  training  PMA  data  by  key  decision  makers  at  the 
TRADOC  schools  is  addressed  more  fully  in  that  section  of  this 
report  beginning  on  page  15. 

Collective  Training  in  Institutions 

There  is  no  collective  training  conducted  in  any  of  the  MOS 
producing  courses  surveyed,  although  some  of  the  survey 
respondents,  especially  those  involved  in  the  maintenance 
courses,  do  indicate  that  in  their  opinion  some  collective 
training  is  in  fact  being  conducted.  This  opinion  is  probably 
due,  in  the  opinion  of  the  authors,  to  a  basic  misunderstanding 
of  the  definition  of  collective  training  on  the  part  of  the 
survey  respondents.  Some  of  the  respondents  do  indicate  that  if 
the  accomplishment  of  a  task  requires  more  than  one  soldier,  they 
view  that  task  as  a  collective  task.  This  is  not  tl._  definition 
provided  in  guidance  from  TRADOC,  however.  TRADOC  Regulation 
350-7,  A  Systems  Approach  to  Training,  defines  a  collective  task 
as  one  which  "requires  echelon  participation  for  its 
accomplishment,"  (p.  45)  snch  as  a  unit  performing  a  task  while 
its  subordinates  echelons  simultaneously  perform  different  tasks. 
The  regulation  further  states  than  "an  activity  that  requires 
more  than  one  person  to  perform  the  same  individual  tasks  for  its 
completion  is  not  considered  to  be  a  collective  task;  e.g., 
lifting  a  load  that  is  too  heavy  for  one  individual."  (p.  45). 
This  would  account  for  the  survey  respondents  claiming  that 
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collective  tasks  are  taught  in  the  courses,  whereas  the  POIs  do 
not  reflect  collective  task  training. 

Although  collective  training  per  se  is  not  reflected  in  any 
of  the  POIs,  some  of  the  POIs  reviewed  allude  to  a  collective 
environment.  For  example,  the  POI  for  the  19E10  and  19K10 
courses  reflect  a  98-hour  field  exercise  in  which  "previously 
learned  individual  skills  are  reinforced  in  a  collective  tactical 
environment  using  MILES  equipment."  In  those  instances, 
approximately  20  individual  skills  are  identified  but  no 
collective  skills  are  specifically  mentioned. 

The  fact  that  the  MOS  producing  courses  do  not  include 
collective  training  (a  situation  which  is  assumed  to  be  the  nonri 
throughout  the  TRADOC  school  system)  is  in  keeping  with 
Department  of  the  Army  level  guidance.  Appendix  A  of  AR  350-1, 
Army  Training,  the  Army's  capstone  regulation  governing  training, 
defines  collective  training  as  that  "training  in  units  to  prepare 
cohesive  teams  and  units  to  accomplish  their  combined  arms 
missions  on  the  integrated  battlefield."  It  would  appear  that  not 
mentioning  collective  training  in  conjunction  with  institutional 
training  is  no  oversight,  thus  leading  to  the  conclusion  that 
Army  policy  is  that  collective  training  is  to  be  conducted  only 
in  the  unit  training  environment. 

There  is  an  element  of  confusion,  however,  with  respect  to 
the  conduct  of  collective  training  in  institutions.  On  the  one 
hand,  the  conclusion  that  must  be  drawn  from  AR  350-1  is  that 
collective  training  is  conducted  only  in  the  unit,  not  the 
institution.  However,  the  TRADOC  Primer,  an  internal  publication 
designed  to  "provide  a  comprehensive  introduction  to  what  TRADOC 
does  and  how  it  runs",  defines  collective  training  as  "training 
either  in  institutions  (emphasis  added)  or  units,  that  prepares 
cohesive  teams  and  units  to  accomplish  their  ....missions." 

Despite  this  apparent  difference  in  perspective,  there  is  no 
documentation  specifying  what,  if  any,  collective  training  is 
conducted  in  institutions.  Accordingly,  there  is  no  formal 
collective  training  PMA  specified  in  any  of  the  documents 
reviewed  during  this  study  effort. 

This  notwithstanding,  the  Armor  School  does  appear  to  get 
involved  in  collective  training  PMA,  and  presumably  the  training 
that  precedes  the  measurement.  As  previously  mentioned,  the 
98-hour  field  exercise  conducted  toward  the  end  of  the  19E10  and 
19K10  courses,  uses  MILES,  which  is  an  engagement  simulation 
system  designed  to  evaluate  collective  performance  in  a 
force-on-force  exercise.  The  use  of  MILES  would  seem  to  indicate 
that,  at  least  at  the  Armor  School,  some  collective  training  is 
provided  in  the  two  courses. 


Integration  Training  fTrainina  of  Individual  Skills  in  Units^ 


Tasks  Trained/Evaluated.  There  is  no  formal  documentation 
as  to  the  number  of  tasks  that  are  trained  and  evaluated  in  the 
integration  training  phase  of  individual  training  in  units. 
Moreover,  none  of  the  respondents  interviewed  during  the  unit 
surveys  was  able  to  state  how  many  tasks  are  trained  in  this 
particular  phase.  In  arriving  at  the  numbers  that  are  reflected 
in  the  matrices  in  Appendix  C,  the  authors  subtracted  the  number 
of  MOS  specific  tasks  contained  in  the  POI  from  the  number  of  MOS 
specific  tasks  shown  in  the  appropriate  soldiers'  manual  or  job 
book  for  Skill  Level  1.  The  result  is  believed  to  be  a 
reasonably  approximate  estimate  of  the  number  of  tasks  that  a 
soldier  needs  training  on  in  order  to  prepare  him/her  to  be  an 
effective  member  of  the  unit,  which  is  the  definition  of 
integration  training.  Table  D-9  shows  the  number  of  tasks  by 
MOS. 


In  two  instances  -  67Y10  and  68B10  -  the  soldier's  manuals 
make  reference  to  an  MOS  training  plan  (MTP)  published  in  the 
trainer's  guide  that  identifies  critical  tasks  for  ea  .h  subject 
area  and  specifies  where  soldiers  are  initially  trained  on  each 
task.  Although  these  trainer's  guides  were  not  available  for  the 
study  effort,  they  would  appear  to  provide  valuable  clues  as  to 
what  the  school  (in  this  case  the  Aviation  Logistics  School) 
views  as  the  scope  of  integration  training.  However,  in  no  case 
in  the  unit  surveys  did  a  respondent,  when  asked  about 
integration  training,  refer  directly  or  indirectly  to  these 
trainer's  guides.  Thus  it  would  appear  that  those  who  would 
benefit  most  from  such  information  may  not  be  receiving  it. 

Although  no  formal  documentation  concerning  the  number  of 
tasks  was  accessed  for  this  effort,  the  survey  respondents  were 
able  to  provide  insights  into  the  extent  and  nature  of 
integration  training  in  their  units.  For  the  most  part,  the 
respondents  indicate  that  the  unit  determines  its  own  integration 
training  requirements,  generally  based  on  the  observed  ability  of 
its  newly  assigned  soldiers.  In  the  case  of  the  maintenance 
related  MOSs,  respondents  are  able  to  indicate  general  areas  in 
which  recent  school  graduates  need  training.  For  example,  survey 
respondents  indicated  that  newly  assigned  soldiers  with  MOS  63E10 
are  weak  in  the  areas  of  wiring,  use  of  wiring  diagrams,  theory, 
and  the  use  of  test  equipment,  thus  implying  that  this  is  tne 
general  thrust  of  their  units'  integration  training  requirements. 

On  the  other  hand,  unit  training  personnel  who  work  with 
soldiers  with  MOS  67Y10  state  that  their  biggest  integration 
training  requirement  is  in  the  area  of  maintenance  paperwork; 
although  they  acknowledge  that  this  area  is  trained  in  the 
schoolhouse,  the  training  is  not  in  sufficient  depth,  it  is 
trained  as  a  separate  task  rather  than  being  integrated  with 
other  training,  and  the  soldiers  receive  it  too  early  in  the 
course,  so  that  by  the  time  they  arrive  in  the  unit,  they  have 
forgotten  what  they  learned.  Survey  respondents  who  are  familiar 
with  integration  training  for  MOS  68B10  soldiers  state  that  the 
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Table  D-9 


Approximation  of  Number  of  Tasks  Involved 
in  Integration  Training,  by  MOS. 


Number  of 
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most  important  area  requiring  integration  training  is  in  the  area 
of  troubleshooting  Talthough  investigation  of  the  POI  at  the 
Aviation  Logistics  School  indicates  that  20  specific 
troubleshooting  tasks  are  trained) . 

Performance  Standards  Used.  While  the  unit  survey 
respondents  seem  uncertain  as  to  the  precise  number  of  tasks 
involved  in  integration  training,  they  do  know  what  performance 
standards  are  used.  In  the  case  of  MOS  19K10,  the  unit  survey 
respondents  indicate  that  they  use  the  standards  contained  in  the 
Soldier's  Manual  and  STP  21-1-SMCT  (Soldier's  Manual  of  Common 
Tasks) .  As  discussed  previously,  these  documents,  in  addition  to 
providing  a  standards  statement  for  each  task,  include  an 
Evaluation  Guide  which  is  a  process-oriented  checklist.  The 
soldier  must  pass  each  step  of  the  test  if  he/she  is  to  get  a  GO 
for  the  entire  task.  Thus  the  standards  used  are  almost 
exclusively  process-oriented.  The  survey  respondents  familiar 
with  integration  training  for  the  maintenance  related  MOSs  - 
63E10,  67Y10,  and  68B10  -  state  that  they  use  the  applicable 
technical  manual  (TM)  as  the  source  document  for  standards.  The 
respondents  further  indicate  that  the  standards  are  procedural, 
step-by-step  checklists. 

Performance  Measurement  Methodology.  The  measurement  of 
performance  related  to  integration  training  is  generally  similar 
across  all  the  MOSs  surveyed.  For  the  19K10  MOS,  task 
performance  is  observed  by  the  soldier's  supervisor,  generally  a 
tank  commander,  during  field  exercises  or  drills,  i.e.,  during 
collective  training  periods.  Thus,  the  training  PMA  is 
relatively  unstructured  and  informal.  In  the  case  of  the 
63E10  MOo,  performance  is  evaluated  by  obseirvation  of  the  soldier 
by  Skill  Level  2  or  Skill  Level  3  mechanics.  Almost  all 
evaluation  is  a  result  of  hands-on  performance  and  is  graded  v.', 
pass/fail.  The  respondents  state  that  subjective  judgment  is 
used  by  evaluators  to  determine  whether  performance  is  a  GO  r  NO 
GO.  MOS  67Y10  soldiers  have  their  performance  measured  against 
the  standards  (the  applicable  -10,  -20,  or  -30  technical  manual) 
and  are  graded  on  a  GO/NO-GO  basis.  The  evaluation,  which  is 
process-oriented,  is  made  by  supervisors  (NCOs)  with  a  checklist 
in  hand.  Respondents  knowledgeable  of  MOS  68B10  indicate  that 
evaluators  observe  the  soldiers'  performance  during  practical 
exercises,  during  actual  job  performance,  and  in  formal 
evaluations . 

In  order  to  facilitate  the  measurement  of  individual  task 
performance,  unit  trainers  are  encouraged  to  use  a  standardized 
form  which  is  contained  in  all  soldier's  manuals.  This  form  (DA 
Form  5164-R)  is  locally  reproducible,  and  unit  trainers  are 
encouraged  to  fill  in  the  procedural  steps  involved  in  the 
accomplishment  of  the  task,  reproduce  the  form  in  the  number  of 
copies  required  to  evaluate  the  soldiers'  performance,  and  check 
off  each  step  as  it  is  performed.  The  form  (which  is  shown  in 
Figure  D-1)  is  designed  to  evaluate  any  individual  training  in 
units,  not  just  integration  training.  Despite  the  availability 
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Figure  D-1.  DA  Forn:  5164-R. 


of  the  form,  none  of  the  respondents  indicate  that  y  use  the 
form  when  evaluating  individual  performance  in  the  ^..xt. 

Sustainment  Training  (Training  of  Individual  Skills  in  Units) 

Use  of  Performance  Measurement  Data.  With  no  exceptions, 
survey  respondents  indicate  that  the  data  resulting  from 
performance  measurement  following  integration  training  are  used 
to  identify  and  diagnose  performance  weaknesses.  This  provides 
the  basis  for  on-the-spot  corrections  of  the  performance 
deficiencies,  counselling  of  the  individual  soldiers,  and 
determining  further  training  requirements  so  that  remedial 
training  can  be  implemented. 

The  term  sustainment  training,  as  used  in  this  effort,  refers 
to  refresher  training  on  individual  tasks  already  learned  by  the 
soldier  either  in  the  institution  (initial  entry  training  for  the 
six  MOSs  selected)  or  in  the  unit  (integration  training) .  It  is 
an  area  that  does  not  lend  itself  to  easy  definition, 
explanation,  or  discussion.  Part  of  the  problem  stems  from  the 
variety  of  publications  which  address  the  training  of  individual 
tasks  in  units  and  the  variety  of  tei-ms  used  to  describe  that 
training.  For  example,  sustainment  training  as  used  in  this 
effort  IS  assumed  to  include  tasks  that  are  found  in  the 
soldier's  manual  for  a  particular  MOS,  the  Soldier's  Manual  of 
Common  Tasks  (STP  21-1-SMCT) ,  the  appropriate  Job  Book,  and  the 
MOS  Training  Plan  (MTP)  contained  in  the  appropriate  Trainer's 
Guide.  The  lack  of  a  single-source  document  for  use  by  the 
trainers  and  training  managers  in  the  units  is  seen  as  a  source 
of  confusion,  especially  since  these  documents  are  not 
necessarily  in  agreement  as  to  what  tasks  the  soldier  is 
responsible  for.  Moreover,  the  taskt.  that  are  included  in  these 
various  publications  are  variously  known  as  common  tasks,  MOS 
skill  level  tasks,  duty  position  tasks,  related  technical  tasks, 
generic  tasks,  and  enabling  skills.  Notwithstanding  these 
problems,  the  unit  survey  respondents  provided  a  significant 
amount  of  information  relating  to  the  area  of  sustainment 
training,  although  it  was  not  possible  to  validate  their 
observations  with  any  official  documentation  or  published 
research. 

Number  of  Tasks  Trained/Evaluated.  There  is  no  document  that 
specifically  identifies  the  number  of  tasks  that  are  to  be 
included  in  sustainment  training  in  units.  In  the  absence  of 
such  documentation,  the  authors  made  the  assumption  that  the 
tasks  listed  in  the  document  (of  those  mentioned  in  the  preceding 
section)  that  contains  the  greatest  number  of  tasks  provides  the 
best  indication  of  what  tasks  the  soldier  is  accountable  for. 
However,  there  is  an  element  of  confusion  associated  with  this 
selection.  The  soldier's  manuals  for  the  two  tank  mechanic  MOSs, 
for  example,  refer  to  duty  position  tasks  and  related  technical 
tasks.  Since  the  manuals  make  the  statement  that  if  the  soldier 
can  do  the  former  tasks  to  stanaard,  he/she  can  successfully 
perform  the  latter  tasks,  there  is  some  question  as  to  the 
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necessity  for  including  those  related  technical  tasks  in  the 
sustainment  training  category. 

Accordingly,  the  number  of  common,  duty  position,  related 
technical,  and  MOS  skill  level  tasks  for  each  of  the  MOSs 
surveyed  is  shown  in  Table  D-10,  along  with  the  source 
document (s)  used  to  arrive  at  those  numbers. 

Survey  respondents  generally  are  unable  to  jtate  the  number 
of  tasks  involved  in  sustainment  training.  For  example,  the 
survey  respondents  involved  with  unit  training  of  MOS  19K10  state 
that  sustainment  training  is  most  likely  to  involve  MOS-specific 
tasks.  In  addition,  all  the  tasks  which  are  included  in 
sustainment  training  are  subject  to  evaluation,  although  none  of 
the  respondents,  nor  any  of  the  documents,  specifically  addressed 
how  many  tasks  are  involved. 

Survey  respondents  familiar  with  the  training  of  MOS  63E10 
soldiers  similarly  are  unable  to  state  how  many  tasks  are 
included  in  sustainment  training  or  how  many  are  subject  to 
evaluation.  Documentation  for  the  MOS  does  not  clarify  the 
issue.  For  example,  the  Soldier's  Manual  (STP  9-63E12-SM)  lists 
64  Skill  Level  1  duty  position  tasks  and  355  related  technical 
tasks,  while  the  Job  Book  lists  78  Skill  Level  1  tasks. 

It  would  appear  that  documents  exist  that  would  clarify  the 
situation  with  regard  to  MOSs  67Y10  and  68B10.  The  documents  are 
the  MOS  Training  Plans  (MTP)  for  these  two  MOSs.  According  to 
the  soldier's  manuals  -  STP  55-67Y12-SM  and  55-68B12-SM 
respectively  -  the  MTP  "identifies  critical  tasks  for  each 
subject  area"  and  "recommends  how  often  soldiers  should  be 
trained  to  sustain  proficiency."  Despite  the  reference  to  these 
MTPs  in  the  soldier's  manuals,  survey  respondents  did  not 
indicate  that  they  have  knowledge  of  them  or  use  them  in  the 
conduct  of  sustainment  training. 

As  mentioned  above,  there  appears  to  be  no  single  source 
document  that  unit  trainers  and  training  managers  can  use  to 
identify  which  tasks  a  soldier  with  a  particular  MOS  is  expected 
to  be  able  to  perform.  Thus  the  implication  must  be  that  unit 
trainers/training  managers  are  required  to  go  to  more  than  one 
source  document  to  determine  what  tasks  are  candidates  for 
sustainment  training  and  evaluation,  thereby  complicating  the 
unit's  sustainment  training  mission.  One  may  assume  that  if  a 
soldier  is  expected  to  be  able  to  perform  a  particular  task,  then 
he/she  can  likewise  expect  to  have  that  performance  measured 
either  during  informal  evaluations  in  the  unit  or  in  the  context 
of  the  formally-administered  annual  Skill  Qualification  Test 
(SQT)  . 

Performance  Standards  Used.  Performance  standards  used  in 
evaluation  of  tasks  included  in  sustainmenL.  training  are  those 
found  in  either  the  appropriate  soldier's  manual  or  technical 
manual.  Whereas  the  standards  statement  is  generally  a 
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Table  D-10 

Estimated  Number  Of  Tasks.  By  MOS  And  Source  Document,  Requiring 
Sustainment  Training. 
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product-oriented  standard  (e.g.,  task  171-122-1041,  Boresight  the 
M240  Coax  machine  gun  on  an  M48A5/M60-Series  Tank,  the  soldier 
has  satisfactorily  performed  the  task  if  "the  center  of  the 
barrel  and  the  center  of  the  infinity  sight  reticle  are  on  the 
same  aiming  point,”  (STP  17-19E1-SM)  (p.  3-62),  the  evaluator  is 
instructed  to  use  the  Evaluator's  Guide  which  is  a 
process-oriented  checklist  to  deteinnine  whether  or  not  the 
soldier  gets  a  GO  on  the  task.  According  to  the  soldier's 
manual,  this  checklist  is  provided  "to  help  the  trainer  conduct 
year-round,  hands-on  evaluations.”  In  the  boresighting  example, 
there  are  31  steps  which  are  evaluated,  each  of  which  the  soldier 
must  pass  to  score  a  GO.  Thus,  regardless  of  whether  the  soldier 
does  in  fact  meet  the  criteria  reflected  in  the  standard,  he/she 
will  get  a  NO  GO  if  any  step  is  failed. 

When  it  comes  to  the  standards  used,  not  all  of  the  MOSs  were 
treated  the  same.  For  MOS  19K10,  the  standards  used  are  those 
that  are  reflected  in  the  appropriate  soldier's  manual.  In  the 
case  of  MOS  63E10,  the  standards  statement  in  the  soldier's 
manual  refers  the  reader  to  the  appropriate  technical  manual 
(TM) ,  e.g.  "you  must  ground-hop  the  power  pack  in  accordance  with 
the  appropriate  publications.”  (p.  2-12).  However,  the  soldier's 
manual  also  includes  an  Evaluation  Guide  with  a  list  of 
Performance  Measures.  Although  the  technical  manuals  were  not 
reviewed  against  the  performance  measures  listed  in  the  soldier's 
manuals,  it  seems  reasonable  to  expect  that  the  performance 
measures  reflect  the  technical  manuals  and  can  be  used  by  the 
evaluator  without  his/her  having  to  refer  to  the  TM  itself.  The 
soldier's  manuals  for  the  two  aviation  maintenance-related  MOSs 
likewise  refer  the  reader  to  the  appropriate  TM. 

Survey  respondents,  in  general,  state  that  the  source  of  the 
standards  used  is  either  the  soldiers'  manual  or  the  technical 
manual.  The  respondents  involved  in  the  training  of  MOS  68B10 
state  that  they  use  both. 

Performance  Measurement  Methodology.  Task  performance  is 
measured  in  at  least  two  ways  in  all  the  MOSs  surveyed.  There  is 
an  informal  evaluation  which  generally  is  conducted  on  a  daily  or 
frequent  basis,  often  in  the  context  of  actual  job  performance  or 
during  individual  or  collective  training  periods.  In  those 
instances,  the  task  performance  is  measured  -  or,  more  accurately 
stated,  evaluated  -  against  the  standards  described  above.  The 
supervisor  or  NCO  observes  the  performance  of  the  task,  in  some 
instances  actually  using  a  checklist  to  record  the  accomplishment 
of  the  procedural  steps  listed  in  the  Evaluation  Guide.  In 
addition  to  the  reproducible  fona  contained  in  all  soldier's 
manuals  (DA  Form  5164-R) ,  the  evaluator  is  encouraged  to  use  the 
applicable  Job  Book  published  by  the  School.  This  is  a  small, 
pocket-sized  book  (several  pages  of  the  Job  Book  for  MOS  19K10 
are  reproduced  below  in  Figure  D-2)  in  which  an  evaluator  can 
record  GO/NO  GO  grades  as  a  result  of  his/her  evaluation  of  the 
soldier's  performance.  Unlike  the  soldier's  manual  (which  should 
contain  the  same  number  of  tasks,  although  this  has  not  been 


D-2  6 


StCTlOK  (I 
OUTT  roSlTIUM  TASKS 
Ml  AiKAnS  Takk  AUTOnoTlve 

ntcHANic/KCCOvcar  timicli  oruATOK 

SKILL  LCVCL  1 

\  r  t  7  .  * 


TASK  NUMBER  AND  TITLE 


GOri?  DATE 


TAMK  coniAT,  losnn,  n\ 


011-4t0-100L 

l«  1 1  fawar  Fack 


01l*410-l001 
Cf*wA4-iU^  Fawaf  f«<k 


•  tWAlO-lQU 

Traaklaikaat  faaVl  Sr«p^*« 

AkafU  •{  Oa««  Nat  Start) 


01l*AfOMOU 

TraakUikaat  fawlt  Sfap^** 
ff««r  L«««) 


0Sl*Af0«n3)0 

AafUca  Cl  a«  traaacKan  i  ea  I  N«l 

S/a  taa 


C\faA  r\ial  Maaala  Ajaaakl/ 


sccTiM  in 

tCLATCO  TtCJiNlOtL  TASKS 
«J(.  Nt  AIAAKS  TAint  AUTOROTirC 

RtcuiatcyaicoyctT  vt^iCLC  otckatok 

SKILL  LCVtL  I 

7  r  1  1  -  r  - 


TASK  NUMBER  ANO  TITLE 


TAJtK,  COnSAr,  lOSM,  Ml 


Gor?  DATE 


Otl*4tD 

-lOO) 

MF^ca 

OU  rtUac 

Afl.LTO 

-1004 

Mflio 

oil  rM«o  AaxakI/ 

9«)*490< 

•190) 

laf  Utt 

Oil  nilar, 

,  on  ^MF  ta  OU 

C»»Uc 

09i«4yi>< 

>1004 

taflaci 

Valva,  Air 

ilta4 

iaaara/i*# ta 1 1  J^alaa  Oil  C«alar 


ffWLTO-lOOf 

taaava/ laa U 1 1  btflaa  Oil  Oaaltr 


Figure  D-3. 


Sample  Pages  from  STP  9-63E-JB. 
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shown  to  be  the  case  in  those  instances  investigated) ,  the  Job 
Book  does  not  contain  a  standards  statement  or  an  Evaluation 
Guide  and  corresponding  performance  measures.  Its  use  would 
appear  to  be  further  limited  by  the  fact  that  there  is  room  to 
record  the  results  of  only  one  iteration  of  the  task  perfotnmance. 

In  the  case  of  MOS  67yi0  as  well  as  68B10,  both  of  which 
relate  to  aviation  maintenance,  there  is  a  process  which  serves 
as  a  more  formal  evaluation  of  individual  task  performance  -  the 
technical  inspection.  Due  to  the  safety  factor  associated  with 
flying,  there  is  a  requirement  for  the  work  performed  by  Skill 
Level  1  mechanics  to  be  checked  by  school-trained  technical 
inspectors.  These  inspectors  must  certify  that  the  work  that  has 
been  performed  on  the  aircraft  or  its  powerplant  has  been 
accomplished  in  accordance  with  the  procedures  specified  in  the 
applicable  technical  manual.  Thus,  in  addition  to  the  informal 
observation  by  the  soldier's  supervisor,  there  is  yet  another 
method  for  evaluating  the  soldier's  performance  of  the  tasks 
related  to  sustainment  training.  While  this  approach  is  the 
ideal  way  of  performing  inspections,  an  educational  specialist  at 
the  Aviation  Logistics  school  had  this  to  say  ,  "No  formal 
requirement  exists  for  relating  Technical  Inspector  (TI) 
inspection  failures  to  performance  failures  or  vice-versa,  unless 
an  individual  TI  takes  it  upon  himself  to  make  the  relationship 
and  follow  through.  Unfortunately,  this  rarely  happens.  The  TI 
is  a  busy  man  in  most  units  due  to  poor  Modified  Table  of 
Organization  and  Equipment  Structure." 

None  of  the  unit  survey  respondents  indicate  that  basic  or 
common  tasks  are  emphasized  in  the  evaluation  of  individual 
performance.  In  every  instance,  the  focus  appears  to  be 
primarily  on  "duty  position"  or  "related  technical",  i.e., 
MOS-specif ic,  tasks. 

For  all  the  MOSs,  there  is  a  structured,  formalized  means  for 
evaluating  the  soldier's  ability  to  perform  the  tasks  associated 
with  sustainment  training.  This  is  through  the  administration  of 
the  Skill  Qualification  Test  (SQT)  which  is  given  annually  to  all 
soldiers  in  the  MOS.  These  tests  formerly  included  a  hands-on 
component  in  which  the  soldiers'  performance  was  observed  in  a 
standardized  test  situation.  However,  due  to  difficulty  in 
developing  reliable  test  instruments,  the  hands-on  component  has 
been  replaced  by  a  test  which  is  strictly  a  paper-based 
evaluation  instrument.  While  performance  of  most  of  the  skills 
involved  in  sustainment  training  across  all  the  MOSs  surveyed 
cannot,  in  the  strictest  sense,  be  evaluated  through  such 
paper-based  tests,  the  SQT  does  provide  a  mechanism  to  evaluate 
the  extent  of  the  soldiers'  knowledge  related  to  the  tasks. 

Use  of  Performance  Measurement  Data.  Without  exception,  the 
personnel  surveyed  in  the  units  stated  their  belief  that  the  data 
that  are  generated  as  a  result  of  training  performance 
measurement  within  the  unit  are  used  by  unit  training  managers  to 
determine  the  remedial  training  required  to  overcome 
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deficiencies.  Even  the  informal  evaluation  of  how  well  a  soldier 
performs  a  certain  task  provides  the  supervisors  an  indication  of 
what  additional  training  is  required  in  order  to  bring  a  soldier 
up  to  the  specified  criterion  level  of  performance.  Similarly, 
the  SQT  results  are  provided  the  supervisors  so  they  can 
determine  training  requirements.  Thus  the  respondents  indicated 
that  the  results  of  the  training  performance  measurement  are  used 
within  the  unit,  although  there  is  some  question  in  their  minds, 
as  discussed  below,  as  to  whether  the  data  are  being  used  by  the 
TRADOC  schools  in  developing  courses  or  training  support 
material . 

Respondents  from  aviation  maintenance  units  also  indicate 
that,  in  addition  to  identifying  training  requirements,  the  data 
generated  from  training  PMA  are  used  to  determine  the  extent  of 
supervision  that  Skill  Level  1  mechanics  need  as  they  perform 
their  jobs.  None  however  states  that  the  requirement  to  have 
technical  inspectors  check  the  work  of  the  Skill  Level  1 
mechanics  is  ever  bypassed. 

Results  of  sustainment  training  PMA,  both  the  informal 
observation  results  and  the  SQT  results,  are  also  provided  to 
the  soldier.  This  feedback  system  gives  the  individual  insights 
into  areas  in  which  he/she  is  weak  and  identifies  areas  where 
additional  training  is  necessary. 

The  Skill  Qualification  Tests  results,  in  addition  to 
identifying  future  training  requirements,  are  also  used  to 
support  various  personnel  actions,  such  as  promotion  and 
reclassification.  Moreover,  SQT  results  are  provided  to  the 
proponent  schools;  TRADOC  Reg  350-7  requires  the  schools  to 
analyze  these  results  to  determine  training  problems  being 
encountered  by  units  in  the  field  so  that  remedies  can  be 
designed  to  overcome  these  problems. 

In  addition  to  providing  training  PMA  data  to  the  individual 
soldiers,  their  supervisors,  and  the  schools  which  have 
proponency  for  the  particular  MOSs,  units  are  encouraged  to  make 
entries  concerning  individual  performance  in  a  Job  Book.  The  Job 
Book  is  designed  to  become  a  part  of  the  soldier's  individual 
personnel  records  and  forwarded  to  the  new  unit  when  the  soldier 
is  reassigned.  Inasmuch  as  this  study  effort  did  not  address 
soldiers  transferring  from  one  unit  to  another,  this  aspect  of 
the  use  of  training  PMA  data  was  not  addressed. 

Finally,  there  is  a  mechanism  whereby  the  TRADOC  schools  are 
required  to  be  proactive  in  seeking  out  training  PMA  data. 

TRADOC  Reg  350-7  requires  the  schools  to  conduct  external 
evaluations  to  determine  "whether  or  not  the  duty  position  or 
mission  performance  requirements  have  been  satisfied  in  the  most 
effective  and  efficient  manner  commensurate  with  the  conditions 
and  constraints  that  prevail  in  the  representative  unit  setting." 
Specific  requirements  include  the  development  of  an 
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evaluation  policy,  preparation  of  an  external  evaluation  plan, 
the  establishment  of  feedback  channels,  the  preparation  of 
statements  of  work  for  contracted  studies,  the  preparation  of  a 
branch  training  team  visitations  plan,  the  collection  and 
analysis  of  data/ informat ion  and  the  preparation  of  evaluation 
reports,  and  the  distribution  of  evaluation  reports  to  the 
appropriate  off ices/agencias  for  action. 

According  to  the  current  TRADOC  school  model,  the  Director  of 
Evaluation  and  Standardization  (DOES)  is  responsible  for 
establishing  these  Branch  Training  Teams  (BTTs)  and  sending  them 
to  units  to  which  soldiers  with  MOSs  for  which  the  school  is 
proponent  are  assigned  in  order  to  determine,  among  other  things, 
the  performance  shortfalls  of  the  graduates.  These  teams  are 
charged  with  accessing  training  PMA  data  as  well  as  subjective 
information  from  unit  trainers  in  order  to  have  a  data  base  on 
which  to  improve  the  quality  cf  institutional  training  as  well  as 
the  training  materials  the  schools  develop  to  support  training  in 
the  units.  Despite  the  existence  of  this  proactive  system,  the 
implementation  of  the  program  appears  to  be  somewhat  lacking. 
Although  the  school  respondents  state  that  the  BTTs  visit  units 
on  a  regular  basis,  none  of  the  unit  survey  respondents  states 
that  he  had  ever  talked  with  anyone  from  the  respective  schools. 
While  many  of  the  respondents  see  the  link  between  the  unit  and 
the  school  as  a  very  important  one,  most  of  the  unit  respondents 
believe  the  link  is  not  as  operative  as  it  might  be.  Again,  the 
conclusions  drawn  may  be  the  result  of  sampling  error,  given  the 
relatively  small  size  of  the  sample  used  in  this  effort. 

Enhancement  Training  (Training  of  Individual  Skills  in  Units^ 

Enhancement  training  is,  for  the  purposes  of  this  study, 
defined  as  training  that  prepares  a  soldier  to  accomplish  tasks 
that  he  has  not  previously  been  required  to  perform.  These  new 
tasks  may  be  required  as  a  result  of  the  introduction  of  new 
equipment,  as  in  the  case  of  the  Ml  tank  replacing  the  M60  tank, 
or  as  a  result  of  changes  in  tactics  or  procedures.  Enhancement 
training  is  also  required  to  prepare  a  soldier  to  perform  at  a 
higher  level,  i.e.,  to  prepare  him  for  promotion. 

Tasks  Trained/Evaluated.  With  respect  to  the  first  category, 
i.e.,  enhancement  training  to  support  a  change  in  equipment, 
tactics,  or  procedures,  there  is  no  list  of  prescribed  tasks. 

The  new  tasks  will  be  dependent  upon  the  new  equipment,  tactics, 
or  procedures.  Until  those  changes  come  about,  the  extent  of  the 
training  required  to  enable  the  soldier  to  perform  the  new  tasks 
cannot  be  determined.  The  required  training  is  developed  by  the 
proponent  TRADOC  school  based  on  the  change;  generally,  if  the 
training  is  occasioned  by  a  change  in  equipment,  a  New  Equipment 
Training  Team  (NETT)  will  be  created  to  accomplish  the  training 
on  site  in  the  unit.  For  training  to  support  changes  in  tactics 
and  doctrine,  the  TRADOC  school  will  generally  develop  a  training 
program  specifically  to  support  the  training  in  units.  In  the 
case  of  preparing  a  soldier  for  promotion  to  the  next  higher 
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skill  level,  the  authors  reviewed  the  appropriate  soldier's 
manuals  and,  where  available,  the  appropriate  job  books  for  each 
of  the  MOSs  to  determine  the  number  of  tasks  Skill  Level  2 
soldiers  were  required  to  perform  that  were  above  and  beyond  the 
requirements  for  SL  1  soldiers.  The  number  of  these  tasks  is 
shown  in  Table  D-11. 

As  can  be  seen,  the  number  of  additional  tasks  that  a  soldier 
at  tlje  next  higher  skill  level  will  be  required  to  perform  varies 
among  MOSs;  moreover,  depending  on  what  source  document  is 
reviewed  the  number  of  additional  tasks  may  even  vary  within  a 
single  MOS.  Thus  for  19E10,  the  Job  Book  reflects  12  additional 
tasks  whereas  the  Soldier's  Manual  reflects  34.  In  the  case  of 
MOS  68B10,  the  soldier's  manual  states  that  "technical  tasks  for 
this  MOS  are  the  same  for  Skill  Levels  1  and  2;"  Table  11  thus 
reflects  ' 0 '  for  the  number  of  tasks  required  at  the  higher 
level . 

Performance  Standards  Used.  The  performance  standards  used 
in  enhancement  training  necessitated  by  a  change  in  equipment 
will  be  reflected  in  the  POI  used  by  the  NETT.  Similarly,  for 
changes  in  tactics  and  procedures,  the  appropriate  TRADOC  school 
will  develop  the  standards  as  part  of  the  training  support 
material.  For  enhancement  training  to  prepare  a  soldier  for 
promotion,  the  standards  are  provided  in  the  appropriate 
soldier's  manual  or  applicable  technical  manual. 

Performance  Measurement  Methodology.  Since  the  training 
programs  that  are  used  for  the  conduct  of  enhancement  training 
are  generally  unique  to  a  special  situation,  there  are  no  data 
available  as  to  the  methodologies  used  in  those  instances  to 
measure  performance.  In  general,  unit  survey  respondents 
indicate  that  observation  by  members  of  the  NETT  or  by  the 
soldiers'  first-line  supervisors  is  the  accepted  methodology. 

For  enhancement  training  to  support  promotion  or  performance  at  a 
higher  skill  level,  respondents  indicate  that  observation  of 
performance  against  the  standards  specified  in  the  soldier's 
manual  or  technical  manual  is  the  primary  means  of  evaluating 
performance . 

Use  of  Performance  Measurement  Data.  Survey  respondents  in 
general  indicate  that  training  PMA  data  generated  through  the 
observation  of  soldiers'  performance  are  used  to  determine  the 
completion  of  training,  i.e.,  when  the  soldier  can  move  on  to 
training  on  the  next  task.  This  is  seen  as  being  the  equivalent 
of  stating  that  the  data  are  used  to  identify  the  necessity  for 
further  training  on  a  task  that  the  soldier  is  unable  to  perform. 

Crew/Team /Scruad  Training  /Training  of  Collective  Skills  in  Units) 

Although  a  significant  portion  of  available  training  time  in 
units  is  devoted  to  collective  training,  the  identity  of  the 
specific  MOSs  is  lost  in  the  training  support  material  developed 
by  the  schools.  For  example,  each  type  of  Table  of  Organization 
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Table  D-11 


Estimated  Number  of  Tasks,  by  MOS  and  Source  Document,  Requiring 
Enhancement  Trdin’ng  for  Task  Performance  at  Skill  Level  2. 


Job  Book  Soldier’s  Manual 


Common 

Tasks 

Duty 

Position 

Tasks 

Related 

Tech. 

Tasks 

MOS 

Skill 

Level 

Tasks 

1 

Duty 

Position 

Tasks 

1 

Related 

Tech. 

Tasks 

MOS 

Skill 

Level 

Tasks 

19E10 

10 

O 

1 

34 

19K10 

Q 

> 

1 

1 

30 

63E10 

22 

31 

78 

31 

78 

63N10 

[NOT  AVAILABLE) 

23 

50 

67Y10 

[NOT  AVAILABLE] 

68B10 

[NOT  AVAilmBLE] 

0 
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Evaluation  Progra  -ARTEP)  which  specifies  the  collective  tasks 
that  the  unit  inu£  oe  able  to  perform  along  with  the  associated 
conditions  and  prescribed  standards.  However,  the  ARTEP  does  not 
specifically  identify  the  soldiers  by  MOS  and  skill  level  who  are 
involved  in  the  collective  task  accomplishment,  nor  which 
individual  tasks  are  associated  v'ith  the  accomplishment  of  the 
collective  task.  Although  this  shortcoming  is  being  addressed  in 
the  Improved  ARTEP  Program  currently  underway  throughout  the 
TRADOC,  the  current  state  of  the  ARTEPs  precludes  the  generation 
of  comprehensive  data  concerning  this  area  of  training. 

Nnmh«a-r  of  Tasks  Trained/Evaluated.  In  view  of  the  data  void 
identified  in  the  preceding  paragraph,  it  is  not  possible  to 
identify  the  number  of  collective  tasks  required  of  each  of  the 
six  MOSs  in  the  unit  training  environment.  Moreover,  since 
soldiers  with  a  particular  MOS  are  assigned  to  different  types  of 
units,  each  of  which  has  its  own  unique  ARTEP,  the  number  of 
collective  tasks  identified  against  each  MOS  would  depend  on  the 
unit  to  which  the  soldier  was  assigned.  For  these  reasons,  no 
data  are  available  which  indicate  the  extent  of  the  collective 
training  requirements  for  the  six  MOSs  surveyed. 

Performance  Standards  Used.  Standards  are  provided  in  the 
ARTEPs  as  mentioned  above.  Although  TRADOC  requires  its  schools 
to  make  the  standards  as  objective  as  possible,  one  of  the 
principal  complaints  concerning  the  ARTEP  is  that  the  current 
standards  are  seriously  flawed.  Among  the  most  frequent 
criticisms  of  the  ARTEP  standards  are  that  they  are: 

•  Compound  standards  -  That  is  they  contain  more  than  one 
criterion,  making  it  impossible  to  assess  task 
performance  if  one  or  more  (but  less  than  all)  the 
criteria  are  failed.  For  example,  if  the  standard 
contains  four  criteria,  and  one  is  failed,  there  is  no 
guidance  as  to  whether  the  whole  task  performance  should 
be  rated  GO  or  NO  GO. 

•  Vague  -  That  is  they  leave  the  evaluator  unsure  of  what 
specific  measures  should  be  applied  to  determine 
successful  task  accomplishment. 

•  They  contain  statements  about  what  to  do  as  opposed  to 
how  well  to  do  it.  For  example,  the  standards  for  a 
particular  task  are  often  in  reality  subtasks. 

•  Subjective  -  They  rely  too  greatly  on  the  subjective 
judgment  of  the  evaluator.  This  is  a  particularly 
serious  criticism  in  light  of  the  frequent  comments 
concerning  the  lack  of  adequate  training  for  the 
evaluator  personnel . 

Performance  Measurement  Methodology.  While  it  is  impossible, 
for  reasons  mentioned  above,  to  isolate  the  training  PMA 
methodology  used  to  measure  collective  task  performance  against 
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each  of  the  six  MOSs  surveyed,  some  comments  can  be  made 
concerning  collective  task  training  PMA  in  general.  The 
evaluations  are  made  either  informally  or  formally,  generally  at 
some  interval  prescribed  by  higher  headquarters.  Informal  ARTEP 
evaluations  can  be  conducted  through  observation  by  the  commander 
at  all  levels  of  command  or  supervision,  such  as  by  the  Battalion 
Commander  down  to  and  including  the  team,  crew,  or  squad  leader. 
These  informal  observations  of  performance,  measured  against  the 
ARTEP  standards,  allows  the  leader  to  identify  performance 
deficiencies  so  that  remedial  training  can  be  scheduled  and 
conducted.  Again,  given  the  nature  of  the  ARTEP  document  itself, 
the  ARTEP  does  not  allow  for  measurement  of  individual  training 
deficiencies,  although  certainly  the  Army  expects  its  leaders  to 
be  aware  that  a  collective  task  performance  deficiency  may  be  the 
result  of  individual  task  performance  deficiencies. 

The  more  formal  ARTEP  evaluations  are  specifically  called  out 
in  a  training  schedule  such  as  put  out  by  the  division  staff. 
Normally,  these  evaluations  are  referred  to  as  •external'  ARTEP 
evaluations,  indicating  that  the  evaluators  come  from  outside  the 
unit  being  evaluated.  As  in  informal  evaluations,  individual 
task  performance  is  not  specifically  addressed,  although  the 
evaluators  are  free  to  mention  such  deficiencies  if  they  are 
contributing  factors  to  the  collective  task  performance 
deficiency. 

Use  of  Performance  Measurer ent  Data.  Although  the  Branch 
Training  Teams  organized  by  each  of  the  TRADOC  schools  is 
encouraged  to  access  all  training-related  data  at  the  units,  to 
include  data  relating  to  collective  training,  the  evidence 
suggests  that  these  data  are  not  getting  back  to  the  schools. 

None  of  the  survey  respondents,  either  at  the  institution  or  the 
unit,  indicated  that  there  is  any  routine  feedback  of  collective 
performance  data  to  the  proponent  school. 

Both  the  informal  (internal)  and  formal  (external)  ARTEP 
evaluations  are  used  to  identify  performance  deficiencies  and 
thus  determine  collective  training  requirements.  Usually,  this 
information  is  generated  during  after-action  reports  following 
the  ARTEP  evaluation,  during  which  the  evaluator(s)  will  discuss 
the  results  of  the  evaluation,  commenting  on  the  unit's  strengths 
and  weaknesses  and  recommending  how  performance  can  be  improved. 

In  addition,  the  results  of  the  evaluations  are  used  to  infer 
the  effectiveness  of  the  unit's  collective  training  program  and 
to  provide  input  to  the  command's  readiness  evaluation  system. 

Advanced  Collective  Training  (Training  of  Collective  Skills  in 
Units) 

The  problems  related  to  the  lack  of  a  cross-walk  between 
collective  task  performance  and  the  task  requirements  associated 
with  each  of  the  six  surveyed  MOSs  is  even  more  pronounced  in  the 
context  of  advanced  collective  training.  In  other  words,  it  is 
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each  of  the  six  MOSs  surveyed,  some  comments  can  be  made 
concerning  collective  task  training  PMA  in  general.  The 
evaluations  are  made  either  informally  or  fonnally,  generally  at 
some  interval  prescribed  by  higher  headquarters.  Informal  ARTEP 
evaluations  can  be  conducted  through  observation  by  the  commander 
at  all  levels  of  command  or  supervision,  such  as  by  the  Battalion 
Commander  down  to  and  including  the  team,  crew,  or  squad  leader. 
These  informal  obseir'atioiis  of  performance,  measured  against  the 
ARTEP  standards,  allows  the  leader  to  identify  performance 
deficiencies  so  that  remedial  training  can  be  scheduled  and 
conducted.  Again,  given  the  nature  of  the  ARTEP  document  itself, 
the  ARTEP  does  not  allow  for  measurement  of  individual  training 
deficiencies,  although  certainly  the  Army  expects  its  leaders  to 
be  aware  that  a  collective  task  performance  deficiency  may  be  the 
result  of  individual  task  performance  deficiencies. 

The  more  formal  ARTEP  evaluations  are  specifically  called  out 
in  a  training  schedule  such  as  put  out  by  the  division  staff. 
Normally,  these  evaluations  are  referred  to  as  external'  ARTEP 
evaluations,  indicating  that  the  evaluators  come  from  outside  the 
unit  being  evaluated.  As  in  informal  evaluations,  individual 
task  performance  is  not  specifically  addressed,  although  the 
evaluators  are  free  to  mention  such  deficiencies  if  they  are 
contributing  factors  to  the  collective  task  performance 
deficiency. 

Use  of  Performance  Measurement  Data.  Although  the  Branch 
Training  Teams  organized  by  each  of  the  TRADOC  schools  is 
encouraged  to  access  all  training-related  data  at  the  units,  to 
include  data  relating  to  collective  training,  the  evidence 
suggests  that  these  data  are  not  getting  back  to  the  schools. 

None  of  the  survey  respondents,  either  at  the  institution  or  the 
unit,  indicated  that  there  is  any  routine  feedback  of  collective 
performance  data  to  the  proponent  school. 

Both  the  informal  (internal)  and  formal  (external)  ARTEP 
evaluations  are  used  to  identify  performance  deficiencies  and 
thus  determine  collective  training  requirements.  Usually,  this 
information  is  generated  during  after-action  reports  followinc 
the  ARTEP  evaluation,  during  which  the  evaluator (s)  will  discuss 
the  results  of  the  evaluation,  commenting  on  the  unit's  strengths 
and  weaknesses  and  recommending  how  performance  can  be  improved. 

In  addition,  the  results  of  the  evaluations  are  used  to  infer 
the  effectiveness  of  the  unit's  collective  training  program  and 
to  provide  input  to  the  command's  readiness  evaluation  system. 

Advanced  Collective  Training  (Training  of  Collective  Skills  in 
Units) 

The  problems  related  to  the  lack  of  a  cross-walk  between 
collective  task  performance  and  the  task  requirements  associated 
with  each  of  the  six  suirveyed  MOSs  is  even  more  pronounced  in  the 
context  of  advanced  collective  training.  In  other  words,  it  is 
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more  difficult  to  associate  the  collective  tasks  for  higher 
echelons  such  as  platoon,  company  and  battalion  with  particular 
MOSs.  For  this  reason,  no  findings  concerning  the  number  of 
tasks  trained/ evaluated,  performance  standards  used,  training  PMA 
methodologies  or  use  of  training  PMA  data  could  be  developed, 
other  than  those  findings  discussed  within  the  context  of 
crew/team/squad  training  described  above. 

Synopsis  of  School  and  Unit  Surveys 

A  replay  of  the  overall  matrix  (Figure  D-3)  is  provided  on 
page  D-37  to  assist  the  reader  in  reviewing  the  matrices  that 
follow.  These  matrices  provide  detailed  synopses  of  the  surveys 
conducted  at  the  schools  and  units  involved  in  this  effort.  Each 
of  the  blocks  provide  a  summary  of  the  comments  made  by 
respondents  with  regard  to  each  of  the  six  MOSs  addressed  in  the 
effort  (arrayed  along  the  vertical  axis)  and  the  categories  of 
training  (arrayed  along  the  horizontal  axis) .  The  purpose  of 
this  section  is  to  provide  elaboration  on  the  basis  for  the 
findings  and  conclusions  presented  in  the  text  of  this  report. 
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I9E10  n60  Armor  Crewman 


19IC10  Ml  Abrams  Armor  Crew. 


63E10  Ml  Abrams  Tank  System 
Mechanic 

63N10  M60A1/A3  Tank  System 
Mechanic 


67VIO  AHI  Attack  Helicopter 
Repairer 

68610  Aircraft  Powerplant 
Repairer 


INSTITUTIOm  ^  ^ _ UNIT _ 

INDIV  ^  ^  »  I  4 _ IW?IVID^AL _ ^  ^  CaiECTIVE 


AIT  /  OStJT 

Integration 

(Initial) 

Training 

S*jslalnmant 

(Refresher) 

Training 

Enhancement 

Training 

Craw/Taam 
Squad  Training 

Advanced 

Collective 

Training 

Individual 

Collective 

Figure  D-3.  Training  performance  measurement  and  assessment 
matrix/study  architecture 
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I  MOS:  1  9E  1 0 

I  Number  of 
I  Tasks  Trained/ 
I  Evaluated: 

I 


Performance 
Standards  Used: 


:  Performance 
'  Measurement 
!  Methodology: 

I 

I 

i 

i 

I 

I 

! 

i  Use  of 
!  Performance 
I  Measurement 
I  Data: 


Source(s)  of 
Information: 


Type  Training:  Training  of  Individual  Skills  in  institutions 

64  discrete  MOS- related  tasks  are  reflected  in  the  Program  of 
Instruction,  these  do  not  include  a  number  of  common  tasks  taught  in 
the  Basic  Training  portion  of  this  OSUT  situation.  A  total  of  6 1  tasks 
are  specifically  evaluated;  survey  indicates  perhaps  as  low  as  50^ 
of  tasks  trained  are  evaluated,  but  this  is  believed  to  refer  to  all  tasks 

POI  refers  to  standards  contained  in  Soldier's  Manual  for  MOS  1 9E 
Skill  Level  1  Survey  respondents  also  indicate  Soldier's  Manual  is 
the  source  of  the  standards  Soldier's  Manual  contains  standards 
( product-oriented)  as  well  as  an  evaluation  guide  which  lists  perfor¬ 
mance  measures  ( process -oriented)  If  soldier  fails  any  step  in  the 
process-oriented  evaluation  guide,  he  receives  a  NO-GO  for  the  entire 
task ,  regardless  of  whether  he  meets  the  standard. 

Examinations  at  end  of  each  major  block  of  instruction  ( 22  hours  are 
hands-on,  2  hours  are  written).  There  are  three  "Gate"  tests,  each  of 
which  IS  an  8-hour  hands-on  performance  evaluation.  Gate  1  marks 
the  end  of  the  BT  phase  of  training.  Gate  2,  and  Gate  3  which  is  the 
end-of-course  comprehensive  test.  All  hands-on  tests  are  scored  Go/ 
No  Go  To  pass,  soldier  must  get  a  "Go "  on  each  and  every  step  involved 
in  the  tas^,  I 

Data  generated  as  a  result  of  performance  measurement  are  used  to 
identify  student  weaknesses  so  remedial  training  can  be  conducted 
Continued  failure  may  lead  to  adverse  personnel  actions,  e  g  reclas¬ 
sification  to  another  MOS.  Data  also  aggregated  to  provide  quality 
control  check  of  instruction.  Must  pass  all  tasks  to  get  MOS. 

I 

Draft  POI ,  Course  Number.  1 9E  1 0-05UT ,  US  Army  Armor  School , 

June  1 985.  Interviews  with  staff  members  of  the  US  Army  Armor 
School  (Dec  85), 
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MOS;  19E10 


Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used: 


Performance 

Measurement 

Methodology: 


Use  of 

Performance 

Measurement 

Data: 

Source(s)  of 
Information; 


Type  Training  Training  of  Collective  Sr  ils  in  .nsttturions 

Not  Identified  POI  reflects  a  98-hour  field  exe'cise  in  whicr.  ' 
"previously  learned  individual  skills  are  reinforced  in  a  collective  • 
tactical  environment  using  MILES  equipment";  21  individual  skills  j 
are  identified  but  no  collective  sk'Hs  are  spec'-'ically  m.ertioned  i 


N/A 


N/A 


N/A 


Draft  POI .  Course  Number  ’  9E  I  O-OSUT ,  US  Army  Armor  School ,  i 
June  1985  Interviews  with  staff  members  of  the  US  Army  Armor  i 
School  (Jan  1 986).  i 
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rMOS:  i9KlO 

I 

!  Number  of 
!  Tasks  Trained/ 
j  Evaluated; 

i 

i 


Performance 
Standards  Used; 

t 

i 

/ 

1 

I 

! 

Performance 
I  Measurement 
;  Methodology; 


Use  of 

Performance 

Measurement 

Data; 

Source(s)  of 
I  Information; 


Type  Training  Training  of  Individual  Skills  m  Institutions 

64  discrete  MOS- related  tasks  are  listed  in  the  POI ,  these  do  not 
include  a  number  of  com.mon  tasks  trained  as  part  of  this  OSUT 
situation  62  of  these  tasks  are  evaluated  A  total  of  5  tasks  are 
reflected  in  the  POi  but  not  tested  See  text  for  further  elaboration  of 
these  and  other  discrepancies  Soldier  must  master  all  tasks  to 
receive  MOS 

^01  "efe's  ’c  Scid-e's  manual  standards  of  performance.  Soldiers 
Manual  for  MGS  i  1 0.  as  well  as  FM  2 1  -  1  ( Common  Tasks  Manual) 
cor  t.ii  's  performance  standards  (product  oriented;  AND  evaluation 
gu’ds  '  process  0''’e'';ted)  if  soldier  faGs  any  oertormance  measure  in 
the  process-onentec  evaluation  guide,  he  receives  a  NO-GO  for  the 
task .  regar  dless  of  whether  he  meets  the  standard 

Soldiers  are  evaluated  bv  oDservation  of  instructors  during  practical 
exercises,  in  addition,  there  are  examinations  at  end  of  each  major 
block  of  instruction  There  are  three  'Gate  tests,  each  of  which  is  an 
8-hour  hands-on  oerformance  evaluation  Gate  i  marks  the  end  of  the 
BT  phase  of  training,  Gate  2,  and  Gate  3  which  is  the  end-of-course 
comprehensive  test 

Data  generated  as  a  result  of  performance  measurement  are  used  to 
identify  student  weaknesses  so  remedial  training  can  be  conducted 
Continued  failure  mav  lead  to  adverse  personnel  actions,  e  g.  reclas¬ 
sification  to  another  M05.  Data  also  aggregated  to  provide  quality 
control  check,  of  instruction 

Draft  PCI ,  Course  Number-  1 9K 1 0-OSUT ,  US  Army  Armor  School ,  June 
1  965  T:"airting  and  Evaluation  Plan  for  the  M  I/M  lAI  Abrams  Armor 
Crev.'.Tian  Course,  US  Army  Armor  School,  Jan  1986  Interview  with 
staff  members  of  the  US  Army  Armor  School ,  Dec  1 985 
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MOS:  19K10 


Type  Training  Training  of  Collective  Skills  in  Institutions 


Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used. 


Performance 

Measurement 

Methodology: 


Use  of 

Performance 

Measurement 

Data: 

Source(s)  of 
Information: 


Not  identified  POI  reflects  a  98-hour  Tankers  Field  Exercise  in 
which  “previously  learned  individual  skills  are  reinforced  in  a 
collective  tactical  environment",  20  individual  tasks  are 
specifically  mentioned  but  no  collective  skills  are  specifically 
mentioned. 

N/A 


N/A 


N/A 


Draft  POI ,  Course  Number  1 9K  1 0-05UT ,  US  Army  APrrior  School , 
June  1 985,  Interviews  with  staff  members  of  th.e  US  Army  Armor 
School.  Dec  1985. 
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MOS:  19K10 


Type  Training  integration  Training  (Training  of  individual  I 
Skill  in  Units)  ' 

Number  of  No  formal  documentation  specifying  number  of  skills  to  be  trained/  i 

Tasks  Trained/  evaluated.  Total  number  of  MOS-specific  tasks  identified  for  soldiers  i 
Evaluated:  with  MOS  19K 10  is  83  Since  school  trains  64  of  these  tasks,  | 

inference  is  that  1 9  tasks  are  candidates  for  integration  training  | 
Unit  survey  respondents  do  not  indicate  knowledge  of  now  many  tasks  1 
are  taught  in  integration  training  i 

I 

Performance  Unit  survey  respondents  indicated  thai  standards  contained  in  Soidis'"':  i 
Standards  Used:  Manual  are  used  However .  Soldier’s  Manual  and  Fm  2 1  -  i  (Cor^mof'  i 
Tasks)  also  contain  Evaluation  Guides  which  are  process-oriented  i 
rather  than  product-oriented.  Using  the  Guides,  if  a  soldier  '"ails  ary 
performance  measure  in  the  guide,  he  receives  a  NO-00  for  tne  entire  i 
task .  regardless  of  whether  he  meets  the  standar  d 

Performance  When  a  soldier  is  assigned  to  the  unit,  he  is  observed  by  h;s  first- ’ir,e  ■' 
Measurement  supervisor ,  a  NCO,  to  determine  the  extent  of  addit'onal  training 
Methodology:  required.  Observation  or  individual  task  performance  normally  tar,es  , 

place  during  field  exercises  or  drills,  i.e.  during  collective  tra’nm'g 
periods.  i 

i 

Use  of  Survey  respondents  indicate  that,  for  the  most  part,  results  of  the  j 

Performance  performance  measurement  are  used  to  determine  future  individual  j 

Measurement  training  requirements.  j 

Data:  j 

Source(s)  of  STP  1  7-  1 9K  l  -SM ,  Soldier's  Manual  for  MOS  1 9K ,  Skill  Level  1  (Sep  i 
Information:  1 984),  STP  21-1  -SMCT,  Soldier's  Manual  of  ComiTion  Tasks,  Sxil  1  | 

Level  1  (Oct  1 985),  Interviews  w'th  unit  trainers  in  various  ! I !  Corps  I 
units,  Ft  Hood.  TX  (Jan  1 986)  I 
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MOS:  1  9K  1  0 

Type  Training  Sustainment  Training  (Training  of  individual 
Skills  in  Units) 

Number  of 

Tasks  Trained/ 
Evaluated; 

A  total  of  223  discrete  tasks  ( Doth  common  and  MOS-specific)have 
been  Identified  for  soldiers  with  MOS  1 9K 1 0.  All  223  are  candidates 
for  sustainment  training,  although  survey  respondents  indicated  the 
MOS  specific  tasks  are  the  most  likely  to  be  trained  in  the  unit.  All 
are  subject  to  evaluation,  but  there  is  no  formal  documentation  of 
which  are  evaluated  in  the  unit 

Performance 
Standards  Used: 

Performance  standards  are  those  that  are  reflected  in  the  appropriate 
Soldier  s  Manual  These  standards  are  generally  product-oriented, 
the  Soldier 's  Manual  also  contains  process- oriented  Evaluation  Guides 
for  each  task  'f  the  soldier  fai's  any  performance  measure  in  the 
Evaluation  Guide,  he  receives  a  NO-GO  for  the  entire  task ,  regardless 
of  whether  he  meets  the  stated  standard. 

\ 

Performance 

I  Measurement 
Methodoiogy. 

Per  formance  is  evaluated  informally  by  observat’on  by  NCOs,  the 
first- line  supervisors  of  the  soldier  In  addition.  Skill  Qualification 
Tests,  primarily  consisting  of  written  components,  are  administered 
annually  to  all  soldiers  with  the  MOS  These  tests  are  prepared  by  the 
proponent  school  and  are  administered  to  all  soldiers  with  the  MOS 
throughout  the  Army. 

Use  of 

Performance 

Measurement 

Data: 

Results  of  informal  evaluations  are  used  to  identify  requirements  for 
future  training  of  the  individual  soldier  SOT  results  are  used  to 
support  personnel  actions,  provide  the  individual  soldier  with  feedback 
as  to  how  well  he  can  perform ,  and  provide  his  supervisors  with  the 
same  information. 

Source(s)  of 
Information 

STP  1 7- 1 9K 1 -SM ,  Soldier's  Manual  for  MOS  1 9K ,  Skill  Level  1  (Sep 

1 984),  STP  2 1  -  1  -SMCT,  Soldier's  Manual  of  Common  Tasks,  Skill 
Level  1  (Oct  1985),  interviews  with  unit  trainers  in  various  III 

Corps  units,  Ft  Hood.  TX  (Jan  1 986) 
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MOS:  19K10  Type  Training  Enhancement  Training  (Training  of  individual  | 

Skills  in  Units)  i 

;  Number  of  If  the  enhancement  training  is  required  as  a  result  of  new  equipment 
Tasks  Trained/  or  other  changes,  the  number  of  tasks  to  be  trained  will  be  specified 
■  Evaluated:  in  appropriate  training  programs  specifically  designed  to  support  the 

change  For  preparation  for  promotion,  the  ^Idier's  Manual  cites 
30  additional  tasks  for  Skill  Level  2  although  the  Job  Book  lists  9 
common  and  1  MOS  skill  level  tasks  at  Skill  Level  2  ! 

Performance  Performance  standard<i  will  be  provided  in  the  sppr  opriate  training  | 
•  Standards  Used:  program  developed  to  suDpo''t  the  enhancement  training.  In  the  case  of  I 
the  training  designed  to  prepare  a  soldier  for  the  next  higher  skill 
level ,  the  standard  is  provided  in  the  Soldier's  Manual. 

I 

Performance  Not  determined  The  methodologies  used  are  dependent  upon  the  unique 
Measurement  training  program  designed  to  support  a  particular  requirement  for 
Methodology:  enhancement  training.  In  the  case  of  enhancement  training  to  progress 

to  Skill  Level  2,  the  Soldier's  Manual  provides  approximately  1 2  per¬ 
formance  measures,  each  of  which  is  required  to  be  passed  in  order  to 
get  a  'GO'  on  the  task. 

:  Use  of  Survey  respondents  indicate  the  data  generated  is  used  to  determine 

'  Performance  when  enough  training  had  been  given 

I  Measurement 
!  Data: 


Source(s)  of  Interviews  with  unit  trainers  in  various  III  Corps  units,  Ft  Hood,  TX 
information;  (Jan  )986);STP  1 7- 1 9K24-SM-TG,  Soldier's  Manual  for  MOS  1 9K, 
Skill  Level  2/3/4  (Sep  1984). 
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MOS:  19K10  TypeTraining  Crew/Team/Squad  Training  C^^a’ning  of  Co’’ec‘ive  ' 

Skills  in  Units 

Number  of  ARTEPs  designed  tor  specific  type  units  identify  ♦he  collective  tasks 
Tasks  Trained/  the  unit  must  be  able  to  perform  but  do  not  specifically  identify  the  ; 
Evaluated;  MOS  and  skill  level  of  the  soldiers  who  are  involved  in  the  perform-  i 
ance  of  the  tasks.  Improved  ARTEPs  will  provide  such  a  crosswalk  i 
Currently.  ARTEP  7 1 -2  (Armor/Mechanized  infantry  Task  Force)  | 
identifies  a  total  of  8  missions  and  76  tasks  for  a-'mor  crews  i 

i 

I 

Performance  The  ARTEP  provides  a  standard  for  each  task  listed  These  standards 
Standards  Used: are  either  quantitative  or  qualitative,  and  involve  extensive  ! 

evaluator  judgment  | 

Performance  Collective  task  performance  is  measured  informal 'v  by  observation 
Measurement  by  unit  leaders,  using  criteria  published  in  the  ARTEP  Training  and 
Methodology;  Evaluation  Outlines.  Formal  ARtep  evaluations  are  made  by  evaluat-  ; 

ors,  usually  from  units  other  than  the  one  being  evaluated,  against 
the  ARTEP  criteria. 

Use  of  Data  generated  as  a  result  of  informal  evaluations  are  used  to  ide'^t'fy 

Performance  weaknesses  and  thereby  determine  collective  training  requirements 
Measurement  Formal  ARTEP  evaluations  are  used  by  the  chain  of  command  to  I 

Data;  determine  training  requirements,  support  readiness  evaluations, 

and  to  determine  the  effectiveness  of  the  unit's  collective  training 
program. 

Source(s)  of  ARTEP  7 1 -2,  Mechanized  infantry /Tank  Task  Force 
Information;  Interviews  with  unit  traine:  s  in  various  III  Corps  units,  Ft  hood,  TX 
(Jan  1986) 
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MOS:  ■9K10  Type  Training  Advanced  Collective  Training  '  Training  of 

Collective  Skills  in  Units) 

Number  of  aRTE^s  designed  for  SDecific  type  units  identify  the  collective  tasks 
Tasks  Trained/  the  unit  must  be  able  to  peri'orm  but  do  not  specifically  identify  the 
i  EvaJuated:  MOS  and  skill  level  of  the  soldiers  who  are  involved  in  the  perform - 

I  ance  cf  the  tasks  imiprovedARTEPs  will  provide  such  a  crosswalk, 

j  Currently,  APJEP  7 ' -2  ( Armor/Mechanized  Infancy  Task  Force) 

i  Identifies  a  total  of  7  missions  and  71  tasks  for  armor  platoons 

I  ( 5  missions  and  56  tasks  for  company  teams) 

I  Performance  The  aPTEP  provides  a  standard  for  each  task  listed  These  standards 
!  Standards  Used:  are  either  quantitative  or  Qualitative,  and  involve  extensive 
!  evaluator  judgment 


j  Performance  Collective  tasK  performance  is  measured  informally  by  observation 

j  Measurement  by  ui^'t  leaders,  using  criteria  published  in  the  arte^  Training  and 

i  Methodology;  Evaluatiori  Outlines  Forma' ARTEP  evaluations  are  made  by  evaluat- 
I  ors,  usually  fromi  units  other  than  the  one  being  evaluated,  against 

the  aRTFR  crite'^ia 

Use  of  Data  generated  as  a  result  of  informal  evaluations  are  used  to  identify 

Performance  weaknesses  and  thereby  determine  collective  training  requirements 
Measurement  Formal  ARTEP  evaluations  are  used  by  the  chain  of  command  for 
Data;  training  requirement  determinations,  support  readiness  evaluations, 

and  to  determine  the  effectiveness  of  the  unit's  collective  training 
orogram 

Source(s)  of  ARTEP  7 1 -2,  Mechanized  mfantry/Tank  Task  Fcce 
Information;  interviews  with  unit  trainers  in  various  III  Corps  units.  Ft  Hood,  TX 
( Jari  1 9S6) 
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MOS;  63E’0 


Type  Training  Training  of  Individual  Skills  in  Institutions 


Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used; 

Performance 

Measurement 

Methodology: 


Use  of 

Performance 

Measurement 

Data; 

Source(s)  of 
Information: 


A  total  of  1 35  discrete  tasks  are  reflected  in  the  Program  of 
Instruction  All  tasks  are  MOS  specific,  i  e  they  do  not  include 
common  tasks  listed  in  FM  2 1  -  1  (Common  Tasks  Manual)  Not 
all  listed  tasks  are  evaluated,  due  to  limited  time  availaDle,  the 
Armor  School  samples  performance  of  some  of  the  tasks, 

ThePOl  refers  to  applicable  Technical  Manuals  for  standards 


5  6  hours  of  written  examinations  ( 70^  is  the  pass-fail 
criterion)  and  62  7  hcursof  performance  examinations 
( without  error  is  the  pass-fail  enter  ion)  are  scheduled  in  POi 
SoiTie  performance  is  evaluated  at  the  completion  of  the  class, 
otherwise  end-of-block  and  end-of-course  evaluations  of 
"selected  critical  tasks"  are  scheduled.  Evaluations  are  made  in 
"sterile"  conditions,  not  the  conditions  expected  to  be  encountered 
on-the-)ob 

Performance  data  are  used  to  identify  student  weaknesses  to 
determine  remedial  training  requirements  and  identiV  lessons 
that  need  modification  ( 20^  failure  rate  gets  special  attention) 
Feedback  data  are  provided  to  DOES  at  USA  Ordnance  Center  and 
School, 

POI  61  1-63E10.  US  Army  Ordnance  Center  &  School.  Nov  1983. 
Interviews  with  staff  members  of  the  US  Army  Armor  School 
(Jan  1986). 
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ttOS:  63E10  Type  Training  Training  of  Collective  m  insti'jticns 

Number  of  No  collective  ski’ls/tasks  are  reflectea  in  the  - 

Tasks  Trained/ 

Evaluated: 


Performance  n/a 
Standards  Used: 


Performance  n/a 

Measurement 

Methodology: 


Use  of  N/A 

Performance 
Measurement 
Data: 

Sourcef  s)  of  POl  6 11  -63E 1 0,  US  Army  Ordnance  Center  5-  S:"oc  .  (  Nla  t  96: 

Information:  interview,  with  staff  me.Tibersof  the  US  Army  a,'"  c'  Scnod  (..'3'' 

1986) 
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I  MOS:  63E10  Type  Training  integration  Training  (Training  of  Individual 
j  in  Units) 

Number  of  No*  dete-rriined  Number  of  tasks  varies  according  to  perceptions 
Tasks  Trained/  of  tracing  requirements  in  units  Survey  respondents  cite 
I  Evaluated:  wmmg.useof  wiring  diagrams,  theory,  and  use  of  test  equipment 

'  as  areas  where  recently  graduated  soldiers  were  weak 


I  Performance  Surver  responce-Ts  state  -ha*  the'/  use  the  applicable  TM  as  a 
Standards  Used:  source  documer*  tor  standards 

;  Performance  Performance  is  evaluated  by  observation  of  the  soldier  by  Skill 
j  Measurement  Level  2  or  Sr'ii  uevei  3  mecnanics  Amost  all  evaluation  Is  a 

I  Methodology:  result  of  nancs-:r,  performance  ana  is  graded  as  pass/fail- 

I  Respcrdents  state  that  subjec'ive  judgment  is  used  by  evaluators 

'  to  deter  mine  whether  performance  IS  a  go  or  no-go  Although 

!  P'*  irstr-r^ients  are  containec  in  Soldiers  Manual,  supervisors 

i  prepare  some  t  20%)  of  the  'nstruments  used 

j  Use  of  LiiTiited  distnbu'mn  Data  are  used  by  NCOs  to  improve 

Performance  miechar.ics  pertcrmance,  i  e  through  counselling  and  on-the-spot 

Measurement  corrections 

Data: 

i  Source(s)  of  mterv.ews  wit,'-.  ur,:t  trainers  :n  various  ih  Corps  unu^,  Fort 
1  Information:  Hood,  tk  (  Jan  '986) 


D-50 


MOS:  e3ElC 


Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used; 


Performance 

Measurement 

Methodology: 

Use  of 

Performance 

Measurement 

Data; 

Source( s)  of 
Information; 


Type  Training  Sustainment  Training  (Training of 
Individual  Skills  in  Units) 

Not  determined.  Soldier's  Manual  lists  64  Skill  Level  (SL)  1 
duty  position  tasks  ( "  a  solid  sample  of  the  kinds  of  tasks  expected 
of  you  on  thejob  ')  and  355  related  technical  tasks  ( "that  can  be 
performed  when  you  meet  the  standards"  on  the  duty  position  tasks. ) 
Job  Book  reflects  78  SL  1  tasks,  while  POl  for  individual  training 
at  the  Armor  School  lists  1 35  tasks. 

Standards  in  Soldier's  Manual  refer  soldier  to  the  appropriate 
Technical  Manual,  eg.  "you  must  ground-hop  the  power  pack  in 
accordance  with  the  appropriate  publications. "  Evaluation  Guide  in 
Soldier's  Manual  provides  a  list  of  Performance  Measur  es,  each  of 
which  IS  graded  pass  or  fail,  and  all  of  which  must  be  passed  in 
order  to  get  a  GO  on  the  task. 

Survey  respondents  state  that  performance  is  measured  generally 
through  observation  by  N(X)s,  and  that  grading  is  GO/NO-GO  if 
any  step  in  process  is  failed,  score  is  NO-GO  in  addition,  soldiers 
are  tested  annually  (written  test)  by  the  SOT 

Feedback  on  performance  measurement  results  provided  to  soldier 
to  Identify  weaknesses  and  additional  training  requirements  NCOs 
determine  future  training  requirements  based  on  performance 
results 

STP  9-63E  12-SM,  Soldier's  Manual  for  MOS  63E,  Skill  Level  1/2, 
(Ap^  1985),  STP  9-63E-JB.  Job  Book  for  MOS  63E  10/20,  Apr 
1 985) ,  Interviews  with  unit  trainers  in  various  1 1 1  Corps  units, 
Fort  Hood.TX  (Jan  1986), 
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MOS:  63E10 


Type  Training  Enhancement  Training  (Training  of 
Individual  Skills  in  Units) 

Number  of  If  the  enhancement  training  is  reguired  as  a  result  nev.  eau’C- 

Tasks  Trained/  ment  or  other  changes,  the  number  of  tasks  to  be  trainee  wil'  be 
Evaluated;  specified  in  appropriate  training  programs  specifically  des’gne-d 

to  support  the  change  For  preparation  for  prorriotion,  the  Soid'er’s 
Manual  cites  3 1  duty  position  tasks  and  78  related  technica' 
tasks  for  SL  2,  primarily  in  the  areas  cf  t’-ouC 'eshoctin;,  ad;ur‘'C, 
and  inspecting 

Performance  Soldier's  Manual  standards,  which  refer  the  soldier  to  tne  appiicac'e 
Standards  Used:  Technical  Manual ,  are  used  Eva'uatior  Guides,  which  e^e  process- 
oriented  performance  measures,  are  also  included  m  the  Solder  s 
Manual. 

Performance  Survey  respondents  state  that  supervisors  (  NCOs)  evaluate 
Measurement  performance  by  oDservation 

Methodology: 


Use  of  Not  determined 

Performance 

Measurement 

Data: 

Source(s)  of  STP  9-63E  1 2-SM ,  Soldier's  Manual  for  MOS  63E  Skill  Levels  1 

Information:  and  2,  Apr  1 985,  Interviews  with  unit  trainers  in  various  i '  i 

Corps  units.  Fort  Hood,  TX  (Jan  1  986) 
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MOS;  63E10 


I  Number  of 
I  Tasks  Trained/ 
j  Evaluated: 


1 

I 

I  Performance 
I  Standards  Used: 


;  Performance 
'  Measurement 
I  Methodology: 


]  Use  of 
j  Performance 
I  Measurement 
I  Data: 

j 

j  Source(s)  of 
!  Information: 


Type  Training  Crew/Team/SQuad  Tra'-ning  ( Training  of 
Collective  Sk  ills  in  Units) 

Not  deteriTiinea  Unable  to  identify  source  docu'Tient  listing 
collective  sk  ills  for  M0S63E10  Survey  resDondents  tend  to  think 
of  collective  tasks  as  those  requiring  more  than  one  soldier  to 
accomplish  ( in  contradiction  with  TRADX  Reg  350-7).  Many 
respondents  view  collective  tasrs  as  SM  tasks  performed  in  a  field 
setting 


Net  detei^m  ^ned 


Not  de’er mined 


Net  oeterminea 


Interviews  with  unit  trainers  in  various  III  Corps  units,  Fort 
Hood,  TX  (Jan  ; 986) 
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MOS;  63EI0  Type  Training  Advanced  Collectible  Training  (Training  of 

(^llective  Skills  In  Units) 

Number  of  Not  determined  No  documentation  apparently  exists  which 

Tasks  Trained/  outlines  ads’Onced  collective  tasks  specifically  for  M0S63E10 

Evaluated; 


Perforiiiance  N/A 
Standards  Used: 


Performance  N/A 

Measurement 

Methodology: 


Use  of  N/A 

Performance 
Measurement 
Data: 

Source(s)  of  interviews  with  unit  trainers  in  various  Ml  Corps  units,  Fort 
Information:  Hood,  (Jan  1 986). 
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MOS:  63N10 


Type  Training  Training  of  individual  Sk’ils  in  Inslitunons  '• 


Number  of  Not  specifically  identified  in  POl,  Number  of  tasks  trained  is  i 
Tasks  Trained/  believed  to  be  similar  to  the  number  of  tasks  for  MOS  63E  10(135 
Evaluated:  discrete  tasks).  All  tasks  are  MOS  specific,  i.e  they  ao  not  include 

common  tasks  listed  in  FM  21-  1  (Common  Tasks  Manual).  Almost 
all  tasks  are  apoarently  evaluated  as  they  are  trained 

Performance  The  POl  refers  to  applicable  Technical  Manuals  for  standards 

Standards  Used:  { 

i 

I 

Performance  6  2  hours  of  written  examinations  (70S  is  the  pass-^ail  | 

Measurement  criterion)  and  63.7  hours  of  performance  examinations  ( withcuf  i 

Methodology:  error  is  the  pass-fail  critenonn)  are  scheduled  m  POl  Almost  all  i‘ 

task  performances  are  evaluated  at  the  completion  of  the  training  ; 
on  the  particular  task ,  with  one  1 6  hour  end-of-course  evaluation  i 
Evaluations  are  made  in  'sterile'  conditions,  not  the  conditions  1 

expected  to  be  encountered  on-the-joD  j 

1 

Use  of  Performance  data  are  used  to  identify  student  wear. nesses  to  j 

Performance  determine  remedial  training  requirements  and  identify  lessons 
Measurement  that  need  modification  (20^  failure  rate  gets  soecial  attention) 

Data:  Feedback  data  are  also  provided  to  DOES  at  USA  Ordnance  Center  and  | 

School.  ‘ 

Source(s)  of  POl  61  1-63N 10,  US  Army  Ordnance  Center  &  Scrool  (May  i?°5),  j 

Information:  Interviews  with  staff  members  of  the  US  Army  Ar^^or  SchooUJan  I 

1986)  ! 
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MOS;  63N10 


Type  Training  Trainirg  of  Collective  Skills  in  Institutions 


j  Number  of  No  collective  skills/tasks  are  reflected  in  the  PO' 

i  Tasks  Trained/ 

I  Evaluated: 

j 


!  Performance  n/a 
[  Standards  Used. 


!  Performance  N/A 
!  Measurement 
j  Methodology: 


Use  of  N/A  ! 

Performance 

Measurement 

Data: 

Source(s)  of  POi  6  l  ’  -63N 1 0,  OS  Army  Ordnance  Center  &  School  (  May  1  985),  j 

Information:  iriterviews  with  staff  members  of  the  US  Army  Armor  School  (Jan  j 

1986). 

_ _ _  . _ _ _ _ _ _ _ J 
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I  MOS:  67Y10  Type  Training  Training  of  individual  Skills  in  Institutions 

Number  of  52  discrete  tasks  are  trained  during  the  course.  Task  performance 

!  Tasks  Trained/  is  evaluated  on  a  sampling  basis  at  the  completion  of  major  blxks 

j  Evaluated:  of  instruction. 

I 

1 

i  Performance  ThePOi  refers  to  "applicable  aviation  maintenance  publications" 

I  Standards  Used;  for  standards.  Survey  respondents  indicate  that  tnose  as  well  as 
;  the  standards  listed  in  the  Soldier's  Manual  are  the  standards 

:  used  by  the  instructors 

i 

,  i 

!  Performance  Survey  respondents  indicate  that  in  the  evaluation  of  task  per-  j 

;  Measurement  formance,  instructors  stress  process  due  to  the  risk  involved  in  | 

Methodology:  flying  and  safety  considerations  instructors  observe  student  per- j 

formance  during  training.  Hands-on  performance  evaluation  is  i 
;  conducted  after  every  major  block  of  instruction  plus  end-of-  { 

‘  course  comprehensive  test.  POI  reflects  49  hours  of  nands-on  i 

!  tests  and  8  hours  of  written  tests.  i 


Use  of  Performance  data  are  used  by  instructors  to  identify  student 

Performance  weaknesses  to  determine  remedial  training  requirements  and 
Measurement  identify  lessons  that  need  modification,  if  students  fail  to  respond 
Data:  to  remedial  training,  they  are  reclassified 


j  Source(s)  of  POi  600- 67Y10,  US  Army  Aviation  Logistics  School  (Aor  1985),  1 

j  Information:  Interviews  with  staff  members  of  the  US  Army  Aviation  Logistics 

i  School  (Jan  1 986) 
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tIOS:  67Y10  Type  Training  Training  of  Collective  Sk'lls  m  institutior 

Number  of  No  collective  skills/tasks  are  reflected  in  ihe  PO! 

Tasks  Trained/ 

Evaluated: 


Performance  n/a 
Standards  Used; 


Performance  N/A 

Measurement 

Methodology: 


Use  of  N/A 

Performance 
Measurement 
Data: 


Source(s)  of  POl  600-67Y10,  US  Army  Aviation  Logistic?  School  (Apr  1955' 
Information:  Interviews  with  staff  menriDersof  the  US  Army  Aviation  Logistics 

School  (Jan  1986) 
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I  MOS:  67Y10 


I  Number  of 
I  Tasks  Trained/ 
I  Evaluated: 


i  Performance 
I  Standards  Used: 

I 

Performance 

Measurement 

Methodology: 

Use  of 

Performance 

Measurement 

Data. 

Source(s)  of 
Information: 


Type  Training  Integration  Training  (Training  of  Individual 
Skills  'p  Units) 

Soldier  s  Manual  (STP  55-67Y 1 2-SM)  refers  to  an  MOS  training 
plan  (MTP)  published  in  the  trainer's  guide  (TG)  that  identifies 
critical  tasks  for  each  suDiect  area  and  specifies  where  soldiers  are 
initially  trained  on  each  task  (N.B.  that  the  Soldier's  Manual  lists 
66  SL 1  tasks,  while  the  PO!  lists  62  tasks,  of  which  1 1  are  not 
trained  to  standard  )  However  ^  survey  respondents  do  not  indicate 
that  they  use  or  have  knowledge  of  the  TG,  Survey  respondents  claim 
that  units  determine  their  own  integration  training  requirements 
and  tnat  maintenance  paperwork  is  their  biggest  training  require¬ 
ment,  that  the  students  get  it  too  early  in  their  MOS  course,  there's 
not  enough  depth,  and  it's  taught  in  a  'vacuum',  so  that  by  the  time 
they  arrive  in  the  unit,  they  have  forgotten  what  they  learned. 

Appropriate  - 10,  -20,  and  -23  technical  manuals  provide  the 
standards  used  by  the  unit 

Peformance  is  measured  against  standards,  and  is  graded  on  a  pass/ 
fail  basis  Observation  is  maae  by  supervisors  (NCOs)  with  check¬ 
list  in  hand  Emphasis  is  on  process. 

Used  by  supervisor  to  identify  performance  weaknesses  and  deter¬ 
mine  training  requirements. 


Interviews  with  unit  trainers  m  various  III  Corps  units,  Fort  Hood, 
TX  (Jan  1966),  STP  55-67Yi 2-SM,  Soldier's  Manual  for  MOS 
67Y,  Skin  Levels  1  and  2 
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1  MOS:  67Y10 

I 

1  Number  of 
Tasks  Trained/ 
!  Evaluated: 


[  Performance 
Standards  Used: 


i  Performance 
Measurement 
I  Methodology: 


!  Use  of 
Performance 
Measurement 
Data: 

Source(s)  of 
I  Information: 


Type  Training  Sustainment  Training  (Training  of  i 

Individual  Skills  in  Units)  i 

t 

Soldier's  Manual  (STP  55-67Y 1 2-SM)  lists  66  Skill  Level  1  i 
tasks  It  also  refers  to  an  MOS  training  plan  (MTP)  published  in  i 
the  trainer's  guide  (TO)  and  states  that  the  MTP  "  identifies  critical  ' 
tasks  for  each  subject  area"  and  "recommends  how  often  soldiers 
should  be  trained  to  sustain  proficiency,"  There  is  no  indication  of  i 
how  many  tasks  are  to  be  evaluated. 

The  Soldier's  Manual  provides  evaluation  guides  "to  help  the  frame' 
conduct  year-round,  hands-on  evaluations,"  Evaluation  guides  are  , 
lists  of  process- oriented  performance  measures  based  on  the 
appropriate  Technical  Manual  —  each  must  be  passed  for  the  sclGie' 
to  get  a  GO  on  the  task .  each  step  must  be  in  proper  sequence. 

Evaluations  are  conducted  "during  individual  training  sessions  or 
unit  collective  task  performance,"  Survey  respondents  indicated 
that  evaluations  are  based  on  quality  and  speed  of  work  and  are  maoe  : 
by  school -trained  technical  inspectors.  Formal  SOT  evaluations  are  ; 
made  annually  ^ 

i 

Performance  data  are  used  to  identify  performance  deficiencies  so  ! 
that  remedial  training  on  tasks  not  passed  can  be  conducted  SOT  j 
data  are  provided  to  supervisors  in  order  for  them  to  make  decisions 
concerning  training  requirements. 

I 

Interviews  with  unit  trainers  in  various  III  Corps  units,  Fort  Hood,  i 
TX(Janl986).  STP  55-67Y 1 2-SM .  Soldier's  Manual  for  MOS  | 
67Y,  Skill  Levels  1  and  2  j 
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MOS:  67Y 1  0 


Type  Training  Enhancement  Training  (Training  of 
Individual  Skills  in  Units) 

Number  of  No  formal  documentation  specifically  addressing  nijrnpe'"  of  tasks 
Tasks  Trained/  for  enhancement  training.  Soldier's  Manual  lists  12  discrete  tasks 
Evaluated:  for  $L  2,  generally  involving  the  replacement  of  components  The 

TG  recommends  a  "strategy  for  training  soldiers  to  perform  higher 
level  tasks.  Respondents  indicate  that  enhancement  training  based 
on  the  introduction  of  new  eouipment  or  changes  ir  procedures  is 
dependent  on  non- standardized  training  programs  developed  to 
support  those  unique  situations 

Performance  case  of  higher  level  tasks,  the  SM  provides  evaluation  guioes 

Standards  Used;  containing  performance  measures  based  on  the  appropriate  tech¬ 
nical  manuals.  These  are  used  as  the  performance  standards  :  n 
training  resulting  from  new  equipment  or  changes  to  oroceoures, 
the  standards  are  reflected  in  the  appropr  late  training  prograr^ 

Performance  Performance  is  measured  agamst  standards  and  is  graded  on  a  pass/ 
Measurement  fail  basis.  Emphasis  is  on  prxess 

Methodology: 

Performance  data  are  used  to  determine  when  training  is  complete 
and  to  Identify  whether  additional  training  is  required 


Source(s)  of  Interviews  with  unit  trainers  in  various  lU  Corps  units,  Fort  Hoed, 
Information;  TX  (Jan  1986),  STP  55-6VYI2-SM,  Soldier's  Manual  for  MOS 
67Y,  Skill  Levels  1  and  2 


Use  of 

Performance 

Measurement 

Data: 
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MOS:  67Y'0 


i  Number  of 
I  Tasks  Trained/ 
j  Evaluated; 

I 

I 


;  Performance 
j  Standards  Used; 

I 

) 

I 

i 

!  Performance 
Measurement 
1  Methodology; 


I  Use  of 
i  Performance 
!  Measurement 
Data; 

5ource(s)  of 
Information; 


Type  Training  Crew/Team/Squad  Training  ( Training  of 
Col’ective  Skills  in  Units) 

Not  determined  uriable  to  identify  source  document  listing 
collective  Skills  for  M0S67Y10.  Survey  respondents  tend  to  think 
of  colIecNve  tas^s  as  those  requiring  more  than  one  soldier  to 
accomplish  ( in  contradiction  with  TRADOC  Reg  350-7)  Many 
respondents  v'ew  collective  tasks  as  SM  tasks  pe'for  med  in  a  field 
seUing 

N/A 


N/A 


N/A 


interv;ev.s  wit-  *ratners  in  va''ious  '!!  Cores  jmts,  Fort 
Hooo,  Tv  ( Jan  I  yS6) 
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MOS:  6'^YIO 


Type  Training  Advanced  Collective  Training  ( T raining  of 
Collective  Skills  in  Units) 


I  Number  of  Not  determined.  No  documentation  apparently  exists  which 
■  Tasks  Trained/  outlines  advanced  collective  tasks  specifically  for  nOSCT’viO 

i  Evaluated; 


Performance  N/A 
Standards  Used; 


Performance  n/a 

Measurement 

Methodology; 


Use  of  N/A 

i  Performance 
J  Measurement 
j  Data; 

!  Source(s)  of  Interviews  with  unit  trainers  in  various  111  Corps  units,  Fort 
:  Information;  Hood,  TX  (Jan  1 986) 
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MOS;  68B  1 0 

Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used: 


Performance 

Measurement 

Methodology; 


Use  of 

Performance 

Measurement 

Data: 

Source(s)  of 
Information: 


Type  Training  Training  o<  Individual  Sr 'Us  in  institutions 

86  discrete  tasks  are  trained  during  the  cou'^se.  'n  addit’C'^ ,  the  POi 
lists  a  total  of  40  "other  tasks  and  subjects  taught  in  resioer,t  train  ¬ 
ing  '  which  are  grouped  under  the  rubric  "EnabSng  Skills  The^e 
IS  no  indicat'on  as  to  now  many  of  the  tasks  are  evaluated 


The  POi  does  not  make  specific  refererice  to  pe''^jr  man'e  -z'incyc^ 
used.  Pesponcents  state  that  the  standar  os  use:  a^'e  deter  r-  med 
by  a  pane!  of  subiect  maHer  e.vbe’'^s  (  S^Es  '  ’  ;sua'h/  MOS-ncinor'c 
who  rotate  m  from  the  field  to  the  scnool ) 


Performance  is  evaluated  generally  foiicwing  every  maior  block 
of  instruction  POI  calls  for  67  hours  of  hands-on  performance 
evaluations  ( 40  of  which  constitute  the  end-ot-course  comiprehen- 
sive  examination)  and  8  hours  of  written  examinations  No  in¬ 
dication  ot  grading  system  used 

Not  determined.  Respondents  indicate  that  DOES  monitors  rest 
results  and  failures,  apparent'y  as  a  quality  contrr.i  check  of  the 
course  itself,  and  to  determine  the  student's  "'■eadiness  '  to  oerform 
in  the  field. 

POl  60  1 -688  1 0,  Aircraft  Powerplant  h=j;,3)rer  Course.  .  SArmy 
Aviation  Logistics  ochocl  (Sep  1984)  interviews  w'lh  sta''^ 
members  of  the  Aviation  Logistics  School  (De:  1966) 
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Type  Training  Training ot  Collective  Skills  in  institutions 


1  MOS:  boB  1  0 

i 

I  Number  of  No  collective  skilH/tasks  are  reflected  in  the  POi 

I  Tasks  Trained/ 
i  Evaluated: 


I 

i  Performance  n  a 

!  Standards  Used: 

1  I 

i  i 

I 

i  Performance  n  a 

i  Measurement 
j  Methodology: 


Use  of  N  A 

I  Performance 
;  Measurement 
I  Data. 

i 

I  Source(s)  of  -Oi  60  l -bSB  1 0,  A'rcraft  Powerplant  Reoairer  Course,  US  Army 

I  Information:  Ay -.cNcr  Lcg's’ics  School  f  Sep  1 954)  Interviews  with  staff 

I  men':De''s  oi  Aviation  Loc’S^'cs  School  ( Dec  1 985) 


D-65  , 


MOS;  68B10 


I  Number  of 
'  Tasks  Trained/ 
j  Evaluated; 

j 

1 

i 


'  Performance 
I  Standards  Used; 

I 

I  Performance 
I  Measurement 
'  Methodology; 


[  Use  of 
I  Performance 
j  Measurement 
j  Data; 

i 

I  Source(s)  of 
i  Information; 

1 


Type  Training  Integration  Training  (Training  of  Individual 
Skills  in  Units) 

Soldier's  Manual  (STP  55-68B 1 2-SM)  refers  to  an  MOS  training 
plan  ( MTP)  published  in  the  trainer's  guide  (TO)  that  identifies 
critical  tasks  for  each  subject  area  and  specifies  where  soldiers  are 
initially  trained  on  each  task.  ( N.B.  that  the  Soldier's  Manual  lists 
1 05  SL  1  tasks,  while  the  POI  lists  86  tasks,  of  which  only  1  is 
not  trained  to  standard.)  Survey  respondents  did  not  indicate  that 
they  use  or  have  knowledge  of  the  TO.  Respondents  also  state  that 
soldiers  arriving  in  their  units  do  not  have  a  good  foundation  in 
troubleshooting,  although  school  POI  refers  to  20  troubleshooting 
tasks  trained  at  the  school 

Survey  respondents  indicate  that  the  appropriate  technical 
manuals  provide  the  performance  standards  used  by  the  unit 

Evaluators  observe  soldiers'  performance  during  practical 
exercises,  during  actual  job  performance,  and  in  formal 
evaluations  Survey  respondents  indicate  that  prxess  is 
emphasized 

Used  by  supervisor  to  identify  performance  weaknesses  and  deter¬ 
mine  training  requirements 


Interviews  with  unit  trainers  in  various  III  Corps  units.  Fort 
Hood,  TX  (Jan  1986),  STP  55-68B  1  2-SM ,  Soldier's  Manual  for 
MOS  68B  .Skill  Levels  1  and  2 


D-66 


MOS:  68B10 


Number  of 
Tasks  Trained/ 
Evaluated: 


Performance 
Standards  Used: 


Performance 

Measurement 

Methodology: 


Use  of 

Performance 

Measurement 

Data: 

Source(s)  of 
Information: 


Type  Training  Sustainment  Training  (Training of 
Individual  Skills  in 


Soldier's  Manual  (STf 


55- 


686  12 


C  f'y*  > 

■ot  .  / 


ISt 


c-i  ■ 


ivSi 


tasks.  It  also  refers  to  an  MOS  training  plan  ( MTP' )  pup  isnec 
the  trainer's  guide  (TG)  and  stale?  that  the  MTP  "loentifies  critical 
tasks  fo''  each  subject  area"  and  "recom, mends  how  cOen  soldiers 
should  be  trained  to  sustain  proficienc'/ "  There  is  no  ind'sa^.’on 
of  how  many  tasks  are  to  be  evaluated 


I 


j 

i 


The  Soldier's  Manual  provides  evaluation  guides  'to  help  me  tra'ner  I 
conduct  year-round,  hands-on  evaluations  "  Evaluation  guides  are  ‘ 
lists  of  process- or  lenied  performance  measures  based  on  the 
appropriate  Technical  Manual  -  each  must  De  passed  for  the  soldier  | 
to  get  a  GO  on  the  task,  moreover  each  step  must  be  in  M.e  c^ope^'  [ 
sequence  Respondents  indicate  that  they  use  technical  roanuals  ana  i 
evaluation  guides  in  the  Soldier's  Manual.  i 

I 

Evaluations  are  conducted  "during  individual  training  sessions  or  | 
unit  collective  task  performance  "  Survey  responcents  indicate  that  ( 
performance  is  observed  by  the  supervisors  ana  that  technical  | 
inspectors  check  to  make  sure  the  task,  has  oeen  performea  to  i 

standard,  in  addition,  formal  evaluations  of  the  ability  to  perform  I 
are  made  through  the  administration  ot  the  annual  ] 

TG  contains  information  so  trainers  can  determine  how  often  soldiers  ; 
need  training  to  ensure  that  they  sustain  proficiency  Respondents  ' 
indicate  that  data  are  used  to  determine  the  amount  of  training  or  i 
supervision  the  soldiers  require  j 

Interviews  with  unit  trainers  in  various  ill  Corps  units,  "ort  | 

Hood,  TX  (Jan  1966);  STP  55-68B  1  2-SM  ,  SoMm^'s  Manua’fcr  j 
MOS  68B ,  Skill  Levels  1  and  2  ! 
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1 

MOS;  6t£’0 

Type  Training  Enhancement  Training  ( ir 
Individual  Sk.ills  in  Units) 

aming  of 

Number  of 

Tasks  Trained/ 
Evaluated; 

1 

1 

1 

i 

I 

No  formal  documentation  specifically  addressing  'dumber  of  tasks 
for  ennancement  training.  Soldier's  Manual  states  that  "technical 
tasks  for  this  MOS  are  the  sarnie  for  skill  levels  ’  and  2"  although 

It  refers  to  a  section  of  the  trainer's  guide  ( 'G)  which  recommends  a 
strateg/ for  training  soldiers  to  perform  nigner  level  tasks.  Respon¬ 
ded  *s  'ndica’e  that  enhancement  training  based  Od  ‘he  introduction  of 
riew  ed'j’omef't  o'  changes  in  proceduf'es  is  dependent  on  non- 
stadcaddized  'raining  programs  developed  to  supbC't  those  unique 
S't.atid^'S.  j 

! 

1  Performance 

1  Standards  Used: 

i 

1 

in  fne  case  of  higher  level  tasks,  there  is  no  mdic 
ped'c'-adice  standards  used  in  training  resulti' 
iTient  or  Changes  'o  procedures ,  the  standards  are 
appropriate  training  program 

o' ion  as  to  the 
'.g  from  new  equip- 
defiecteo  in  the 

i  Performance 
Measurement 
Methodology: 

Not  laentifiec 

Use  of 

Performance 

Measurement 

Data; 

Not  Identified. 

Source(s)  of 
information: 

Interviews  with  unit  trainers  in  various  IN  Corps  jnits,  Fort 

Hood,  TX  (Jan  !986).  STP  55-66B  i  2-SM ,  Soldier's  Manual  for 

MOS  68B ,  Skill  Levels  1  and  2 
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I  I 

)  MOS:  68B10  TypeTrainlng  Crew/Team/Squad  Training  (Training  of 
j  C^onective  Skills  in  Units)  j 

i  I 

I  Number  of  Not  determined  Unable  to  identify  source  document  listing  j 

j  Tasks  Trained/  collective  skills  for  MOS  68B 1 0  Survey  respondents  tend  to  think  ( 

i  Evaluated:  of  collective  tasks  as  those  requiring  more  than  one  soldier  to  I 

j  accomplish  (contradicting  TRADX  Reg  350-7).  Respondents  view 

.  collective  tasks  as  SM  tasks  performed  in  a  field  setting 

i  ■  I 

1  I 

Performance  n/a 

I  Standards  Used;  I 

i  I 


I  Performance  n/a  I 

;  Measurement  ! 

Methodology:  i 

I  j 

i  i 

I  i 

j  Use  of  N/A  j 

i  Performance  I 

;  Measurement 
Data: 

! 

Source(s)  of  interviews  with  unit  trainers  in  various  111  Corps  units,  Fort 
Information;  Hood,  TX  (Jan  1 986).  i 
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MOS:  68B10  Type  Training  Advanced  CoMect've  Tre-ning  CT-aimng  cf 

Collective  Skills  in  Units) 

Number  of  Not  determined.  Nod  documentation  apoarently  exists  which 

Tasks  Trained/  outlines  advanced  collective  tasks  SDecifically  for  M0S6SB10 

Evaluated: 


!  Performance  n/a 
I  Standards  Used: 


Performance  N/A 

Measurement 

Methodology: 


Use  of  N/A 

Performance 
Measurement 
Data: 


Source(s)  of 
Information: 


Interviews  with  unit  trainers  in  various  U':  Cc"ps  units,  i^ort 
Hood.  TX  (Jan  1 986) 


APPENDIX  E 

KEY  DECISION  MAKER  QUESTIONNAIRE  I 


E-1 


This  questionnaire  was  developed  by  the  U.  S.  Army  Research 
Institute  to  determine  key  decision  maker  Information  needs 
In  relation  to  training  measurement,  assessment,  and  cost 
effectiveness. 

A  space  Is  provided  for  your  name.  If  you  would  rather  remain 
anonymous,  feel  free  to  do  so.  When  you  have  finished,  fold 
this  paper  so  that  the  address  Is  visible  and  place  It  In  the 
mall  box  -  no  stamp  Is  necessary.  Thank  you  for  your  cooperation. 

Name _ Position _ 

Experience  _ _ 


P-ART  I 

1.  WHO  IN  YOUR  UNIT  makes  decisions  on  training  performance 
measurement  and  assessment? 


2.  HOW  OFTEN  do  your  decisions  affect  the  following?  [Please  Indicate 
your  answers  by  placing  a  checkmark  In  the  appropriate  box.) 


E-2 


3a.  What  kind  of  Information  do  you  need/get  to  make  your  decision? 
[Please  Indicate  your  answer  by  placing  a  check  mark  under  the 
appropriate  columns.] 


DATA 

Do  Yoi 
This 

j  Need 
Data? 

ISX£ 

»nnrj 

Is  It  Qua 
or  Quan 

Iltative 

Htatlve 

Yes 

No 

Yes 

No 

Qua! 

Quan 

1.  Feedback  from  within  unit 

2.  Training  Effectiveness  Data 

3.  Feedback  from  Institution 

4.  Requirements  of  Own  Unit 

5.  Cost  Effectiveness  Data 

6.  Requirements  of  Higher  HQs 

7.  SOT  Results 

8.  ARTEP  Results 

9.  Knowledge  of  Learning  Rates 
to.  Tng  Device  Effectiveness  Data 

_ 

1 

3b.  If  you  need  other  Information  not  listed  above,  then  list  other  types 
of  data  and  Indicate  If  you  get  this  data  by  placing  a  check  mark  under 
"YES"  or  "NO"  column. 


Do  You  Get 

This  Data? 

Is  Tt  Qualitative 
or  Quantitative? 

Yes 

No 

Qual  Quan 

LIST  OTHER  TYPE  DATA 
YOU  NEED 


PART  li 


Please  indicate  how  important  you  believe  the  data  you  selected  for 
questions  3o  and  3b  are  when  you  make  decisions  about  training 
measurement  end  assessment. 

First.  SELECT  10  types  of  dote  you  believe  ere  most  important  when 
making  decision. 

Second.  RANK  ORDER  you  selections.  Let  1  indicate  ‘very  important" 
and  10  indicate  "least  important". 

Third.  WEIGH  each  selection  by  indicating  with  percentages,  how 
much  value  you  attribute  to  each  factor  when  making  your  decisions. 
Please  make  sure  that  the  percentages  add  up  to  1 0OZ. 

For  example,  if  you  select  "SOT  results"  as  one  of  the  10  types  of  data 
you  believe  are  most  important  when  making  decisions  about  training 
measurement  and  assessment,  prioritize  it  In  order  of  importance  and 
assign  a  percentage  to  indicate  how  important  it  is  when  making  your 
decisions. 


List  and  Rank  Order  Your  Selections  Weight  ( Z) 


TOTAL  =  100Z 
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PABim 

PLEASE  USE  THE  FOLLOWING  SPACE  TO  WRITE  ADDITIONAL 
COMMENTS  YOU  WOULD  LIKE  TO  MAKE. 


This  questionnaire  was  developed  by  the  U.  S.  Army  Research 
Institute  to  determine  key  decision  maker  Information  needs 
In  relation  to  training  measurement,  assessment,  and  cost 
effectiveness. 

A  space  Is  provided  for  your  name.  If  you  would  rather  remain 
anonymous,  feel  free  to  do  so.  When  you  have  finished,  fold 
this  paper  so  that  the  address  Is  visible  and  place  It  In  the 
mall  box  -  no  stamp  Is  necessary.  Thank  you  for  your  cooperation. 

Name _ Position _ 

Experience  _ 


PART  I 

1.  WHO  AT  THE  SCHOOL  makes  decisions  on  training  performance 
measurement  and  assessment? 


2.  HOW  OFTEN  do  your  decisions  affect  the  following?  (Please  indicate 
your  answers  by  placing  a  checkmark  in  the  appropriate  box.] 
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3a.  What  kind  of  Information  do  you  need/get  to  make  your  decision? 
{Please  indicate  your  answer  by  placing  a  check  mark  under  the 
appropriate  columns.] 


Do  You  Need 
This  Data? 


1.  Feedback  from  Active  Units 

2.  Training  Effectiveness  Data 

3.  Feedback  from  Instructors 

4.  Tng  Device  Effectiveness  Data 

5.  Cost  Effectiveness  Data 

6.  DOES  Data 

7.  End  of  Cycle  Test  Results 

6.  Test  Validity  &  Reliability  Dati 

9.  Knowledge  of  Learning  Rates 

10.  SOT  Results 


3b.  If  you  need  other  information  not  listed  above,  then  list  other  types 
of  data  and  indicate  if  you  get  this  data  by  placing  a  check  mark  under 
■'YE5“  or  ‘NO"  column. 


PART  [l 


Please  Indicate  how  Important  you  believe  the  data  you  selected  for 
questions  3a  and  3b  are  when  you  make  decisions  about  training 
measurement  and  assessment. 

First.  SELECT  10  types  of  data  you  believe  are  most  Important  when 
making  decision. 

Second,  RANK  ORDER  you  selections.  Let  1  Indicate  ‘very  Important* 
and  10  Indicate  ‘least  Important*. 

Third,  WEIGH  each  selection  by  Indicating  with  percentages,  how 
much  value  you  attribute  to  each  factor  when  making  your  decisions. 

P  lease  make  sure  that  the  percentages  add  up  to  100S. 

For  example.  If  you  select  ‘SOT  results*  as  one  of  the  10  types  of  data 
you  believe  are  most  Important  when  making  decisions  about  training 
measurement  and  assessment,  prioritize  It  In  order  of  Importance  and 
assign  a  percentage  to  Indicate  how  Important  It  Is  when  making  your 
decisions. 


List  and  Rank  Order  Your  Selections 

Weight  (X) 

1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

_ 

TOTAL  =  100* 
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BART  U1 


PLEASE  USE  THE  FOLLOWING  SPACE  TO  WRITE  ADDITIONAL 
COMMENTS  YOU  WOULD  LIKE  TO  MAKE. 


E-9 


APPENDIX  F 

KEY  DECISION  MAKER  QUESTIONNAIRE  II 
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